Archive for the ‘Usenet’ Category

yEnc Decoder compliance

Tuesday, August 26th, 2008

Not all yEnc decoders are equal.  Is your yEnc Decoder doing everything it should?  Or is it doing the bare minimum?

yEnc has more advantages than simply being smaller in size.  yEnc also ensures that the file has been delivered intact via two methods:

  • CRC32 Error Checking
  • File Size Checking

If you yEnc decoder doesn’t support the above features of yEnc, not only are you losing out on some of the benefits of yEnc, but your yEnc decoder is not even yEnc compliant.

CRC32 Error Checking

yEnc CRC32 error checking uses a technology similar to what CDs and DVDs use to check for errors.  It uses a mathematical algorithm at creation time to generate a checksum value based on the data and stores that checksum on the data.  At play time, the value is recomputed against the data and compared against the stored value.  If the checksums do not match, then an error has occurred while reading the data.

In order for a yEnc Decoder to be compliant with the latest yEnc standards, a yEnc Decoder must include CRC32 error checking.  All yEnc attachments have the CRC32 checksum, and if your yEnc Decoder isn’t evaluating the checksum, then you won’t know when you’ve downloaded a file that has been corrupted or modified since it was created.

You may download that treasured song that you’ve been looking for, then when you go to play it back, it may not play or it may contain defects.  At that point, it may be too late to try to download the song again from another source.

Embedded yEnc Decoders will not provide CRC32 error alerts.  yProxy notifies you immediately when an attachment with an error has been detected.  yProxy beeps and changes the system notification icon to one with a large red exclamation mark, in addition to logging the error.  None of yProxy’s competing yEnc proxies provide CRC32 error detection.

File Size Checking

The file size is also computed at creation time and stored in every yEnc attachment.  All yEnc compliant yEnc Decoders must check the final file size against the stored value.  If a file has been truncated by the news server or parts of it are missing, the file may be unusable.  It is important for your yEnc decoder to be able to determine the difference between a corrupt file and an incomplete file.

If the file is incomplete, you may simply need to try again later after the file has finished propagating.  An incomplete file may still be usable, depending on the type of file.  An MPEG video file with the last second missing will still be playable, and you may not notice that missing second.  Therefore, it is important for your yEnc decoder to distinguish between a corrupted file and an incomplete file.

yProxy notifies you immediately if the downloaded file size does not match the expected size.  yProxy will beep and place an exclamation mark in the system notification icon, in addition to logging the error.  The error displays the expected file size and the actual file size so that you can determine the extent of the problem.

Embedded yEnc decoders cannot check the file size and none of yProxy’s competing yEnc proxies provide file size checking.

yEnc Decoder Compliance

CRC32 error checking and file size checking are valuable features of yEnc encoding.  yEnc decoders must support both CRC32 error checking and file size checking in order to be compliant with the latest yEnc standards.  yProxy is the only yEnc decoder proxy that is yEnc compliant.

 

yEnc saturation on Usenet for binary content

Thursday, August 7th, 2008

It’s becoming more and more rare to see quality binary attachments posted that are not in yEnc format.

We are also seeing the number of Usenet groups that have fresh content decrease, as supergroups evolve that contain the majority of Usenet’s quality content.

Usually, yEnc will take over an entire news group, and all downloaders of that group end up having to find a yEnc decoder solution.

Most high volume groups transition to yEnc rather quickly.  Therefore, due to the high volume posted to these supergroups, and the decreasing number of groups, the percentage of content that is posted using yEnc is very high.

Interestingly, spam is typically not posted with yEnc.  Spammers want to reach the largest audience, so they don’t want to exclude those without yEnc decoders.

Therefore, yEnc use is also becoming a good indicator of legitimate content.  Content that is not yEnc encoded is more and more likely spam.  New posters, or infrequent posters, also may not have the proper tools to post in yEnc, but the quality of their content is marginal for most categories.

Even the yEnc test newsgroup will sometimes have some surprisingly good content, and it’s mostly barren of spam.

As the ability of news servers to filter out spam increases, and the amount of spam on peer to peer systems increases, Usenet is making gains over other file sharing methods.

I believe that the popularity of Usenet is likely to spike up in the near future, barring unforeseen, new alternatives.  In addition, yEnc usage will continue to increase until it has mostly saturated the binary newsgroups.