English language English

Usenet Binary Files Explained

Usenet Binary Files

Usenet binaries are a type of file that is widely used in the Usenet community. Usenet binaries take up a good majority of the Usenet information out there. A binary file is basically a file containing binary data, random bytes or words, as opposed to a text file, containing only printable characters.

Common classes of binary files are programs in machine language ready to load into memory and execute. The Usenet binaries however are larger files such as images, .exe’s, or multimedia files such as Divx and AVI. These larger files are broken down into smaller files making up Usenet binaries. The binaries are then broken up into these files by different programs and have to be put back together by the same program that usually breaks them down. For instance, the program WinRAR uses .rar as an extension on its files and the number it assigns to each part is the order, in which they need to be put back together. Also popular newsreaders such as SABnzbd and NZBGet combine multipart files automatically.

Usenet binaries are a very helpful and effective way to transfer large files in the Usenet community. If it weren’t for binaries in Usenet, you would only be able to see text files in newsgroups. The problem with binaries is that if one of the files is missing, it will be hard to combine the files. An extra program that can use par files to replace the missing .rar’s can be used and will usually work for that purpose.

History of Binary Files on Usenet

A short time after people started utilizing Usenet as an all-text discussion forum, the users of certain newsgroups thought that this system would be ideal to share files with each other. However, the newsgroups were not designed to transfer binary files – they could only transfer text files. Users then thought of taking a binary file and converting it (encoding it) so that it is becomes a text message. That enabled users to distribute those text messages (containing files) throughout the newsgroups in Usenet, and then whoever wanted the file, could just download the message and convert it back (decode it) into the original binary file.

The newsgroups operate in almost the same way today. Of course, there have been great improvements in the software used to download the messages from the news server to the individual’s computer that makes getting files from the newsgroups a lot easier than it was in the beginning. A typical large file from a binary newsgroup is usually split into many pieces, as there is a limit to the file size that can be inserted in a text message. Many news servers will not accept messages that are longer than 10,000 lines, many even less than that.

That is why files need to be split up in many parts and each part needs to be posted separately on the server. Programs like WinZIP and WinRAR can split files in equal parts (and a reminder) for you. The RAR/ZIP files are then split into text messages, each one containing the maximum number of lines (or less). These are then posted to the newsgroup. It may sound complicated but it is actually a very efficient way to post large files, spreading them out in different articles. Similarly, the messages can be downloaded and put back together by newsreaders in the same manner, resulting in the original file that was posted.

What are Multi-Part Binaries?

Most of the binary files found on Usenet are what are considered multi-part binaries. The reason they are called this is due to the fact the files must be split up by the people posting the messages, as most news servers will not accept files with more than 20,000 lines. This means that a 700MB file will often be split up into 30 to 40 parts so that the news servers will accept it. In order for you to use the file on your computer you will need to download all 30 or 40 parts of that file. Newsreaders like SABnzbd make it very easy for you to do that.

Combining Binary Files

Have you even wondered why it is that on Usenet you cannot simply download usable files in one step like you can with other programs? You have probably noticed that files are usually posted as multiple small files and you may be confused what to do with them and how to make them work once you have downloaded all the parts.

First off, I would like to explain why files are posted as multiple parts on Usenet. The first reason is because most Usenet providers limit the maximum size of articles that can be posted. This is because Usenet was originally to transport only small text messages and it is not well equipped to handle large files. If you attempted to post a large file as one part, your news server would simply reject it.

The second reason is for convenience. Many internet users still have low speed connections. Breaking your binary files into multiple small files allows such users to download only the parts that they do not already have. This is handy if their connection was dropped while downloading.

A third reason is data integrity. When you use a program to break a binary into multiple smaller files, parity data is added. This helps to prevent data corruption that would cause files such as executable binaries not to run. When recombining the file, the decompression utility will warn you if one or more of the files is corrupted, thus enabling the user to download those parts again, from another server if necessary. If the file had not been posted as multiple parts, the entire file would be useless.

Now that you know why Usenet binaries are posted as multiple parts, I will explain how to handle them. There are two options. The first is to install a newsreader program that has encoding and decoding technology built in. One such program I suggest is GrabIt. This type of newsreader does all of the decoding work for you.

If you are already comfortable with a newsreader that does not have built in decoding technology, you will need to use a separate program, such as WinRAR, to recombine the numerous parts for you. To proceed, simply download all of the file parts into a directory and show the decoding program where they are. It will then recompile the binary file back into its original, usable state. Once this is done, you can safely delete the file parts you downloaded before.

Downloading Binaries with a Newsreader

Downloading binaries off of Usenet is best achieved with a news reader such as SABnzbd, NZBGet, News Bin Pro or GrabIt. A news reader is a must because all of them are capable of splitting and combining larger parts of files. It will come down to personal preference when picking out a news reader because they all do the same job. Basically, the combine and decode the multipart binaries so that you can view them.

As Usenet relies on the NNTP protocol for maintaining orderly communications between peers, it is more or less chained to this technology. As a result, newsgroup binaries must be encoded as clear text and transmitted to the servers this way. This is extremely inefficient from bandwidth and speed point of view as downloading a newsgroup binary file of 1MB in fact requires transferring roughly 1.25MB.

Often the newsgroup binaries posts on Usenet are archived in order to save bandwidth to the poster. In addition, most Usenet providers allow only files smaller than 100MB on their networks so posters split even further the archives using specialized posting software programs to ensure they propagate around the world seamlessly. Combining multiple parts of newsgroup binaries in order to obtain useful information may not be fun but it sure is worth the effort! Despite, the setbacks Usenet is facing as a result of its foundations, it is an excellent environment for sharing newsgroup binaries. From videos to programs, anyone is able to post his or her creation and present it to the World.

Encoding Binary Files

When users post the binary file, they must be sure to encode the binary file. What this “encoding” does, is it turns the binary file into text so that the server will accept it as a valid file. When users download these files, they have to decode the message from text back to its original binary file so that they have the file they wanted. Today there are many newsreaders that will do both the encoding and decoding of these files for you.

There are many users that argue that binary files are not what Usenet is about, and it destroyed the image of Usenet. On the other hand, binary files are what allowed Usenet to develop into what it is today: a strong, growing, and thriving community of peers that are able to exchange information in the form of text and binaries.

Binary File Extractor (Decompression)

A binary decompress or, also known as a binary compressor (since it does both compressing and decompressing), is simply a binary encoding/decoding program such as WinZip or WinRar. Those software applications are generally known as file compression programs in layman’s terms but the process is more commonly known as encoding and decoding in the Usenet community.

The goal of this is to combine the parts of multi-part binary files that have been posted in Usenet, in order to get one big comprehensive, meaningful file that can be played, watched, or executed, depending on the type of file it is. Most commonly, large movie, mp3, software, or game files need to be broken down in multiple smaller parts in order to be posted and then downloaded in Usenet. This is needed because the w messages have a limit to their size and cannot hold the entire big files.

Breaking files down and recombining them, however, is no big deal, with the use of a binary decompressor/compressor. You just basically need to select the files that you wish to compress or decompress and the binary decompressor will do the rest for you. Some newsreaders have this technology built in them but it is generally a good idea to have at least one binary decompressor around. Plus, you can get the most popular binary decompressors from the Internet for free so it doesn’t hurt to download one if you are into posting and downloading huge binaries from Usenet.

PAR Files on Usenet

Par (Parity Volumes) file are files used to repair broken archive files. For example let’s say you download a file from Usenet that has 2,000 parts. If 1 of those parts is broken you won’t be able to combine the file and use it. That is were PAR files come into play. You can use the PAR files to repair the 1 broken piece and then combine the file successfully. Par files can also be encoded and in multiple parts so you’ll need to download all the parts then decode and combine them.

PAR File Software

Most of the popular newsreaders out there that are dedicated to binaries already support PAR files. Newsreaders such as sabNZBD and NZBGet perform post processing after downloading from the newsgroups. During post processing combining of the files occurs. If those newsreader find that a block is missing they will use the included PAR files to repair the missing block. If you are using a newsreader that doesn’t support PAR files you can use a program called QuickPar to repair broken files. You can download QuickPar from quickpar.org.uk.

Providing Quality Usenet Service for Over 10 Years!