3 ms·

Do you mean newsgroups like alt.binaries.* ? I completely forgot about that. They ate up all the bandwidth? I'll take your word for it. Source code sharing wa

by Isamu 7y ago

Do you mean newsgroups like alt.binaries.* ? I completely forgot about that. They ate up all the bandwidth? I'll take your word for it. Source code sharing was pretty important I remember. That was how I first got Perl source code to compile, although it was probably not in the alt newsgroup tree, it must have been in some other that escapes my memory.

tptacek 7y ago
They ate up disk and CPU. It was the nature of NNTP, at least at the serious provider levels, that you had to be able to keep up in real time with a certain high level of responsiveness, or you'd miss your window for posts and there'd be holes in your binaries (because Usenet is perhaps the dumbest imaginable way to transmit a large binary, a Usenet binary was a chain of separate posts, all of which were needed to reassemble the binary). You could run a Usenet server without binaries, just to host the discussions and source code sharing and whatnot. But customers would have none of it: if you didn't have binaries, you weren't serving real Usenet, and they'd go to ISPs that did, and when ISPs generally stopped serving Usenet because it had become the world's lamest warez and porn distribution network, they moved to centralized NNTP services.
- webmaven 7y ago
  > (because Usenet is perhaps the dumbest imaginable way to transmit a large binary, a Usenet binary was a chain of separate posts, all of which were needed to reassemble the binary) I just realized that this is superficially similar to how bittorrent works...
  - tptacek 7y ago
    Not at all. BitTorrent is forward error corrected. It splits binaries up into chunks as an optimization, and ensures that you don't need a precise sequence of chunks to reassemble the file; an error correcting code ensures that you only need k of n chunks. Usenet binaries were literally just raw binaries, uuencoded(!), and split up into n chunks, of which you need all n chunks. Those streams of chunks were then broadcast to every node on the network, despite the fact that only a subset of nodes ever wanted any particular binary. It was and remains the most batshit file transfer mechanism ever devised.
    - lvh 7y ago
      Not that I'd know: but at some point after the nineties Usenet figured out PAR files too, so you wouldn't quite need all the chunks because NNTP servers would drop some of them occasionally -- though to your point, yes, every NNTP server would de facto mirror everything.
    - 300bps 7y ago
      They actually fixed that problem around 2001! It's been over 15 years since I've done anything with Usenet binaries so I'm shocked I was able to recall the PAR file technology. Essentially, if you posted 50 parts of your encoded binary, you would generally post an additional 5 or 6 PAR files. Downloaders could then utilize a PAR file on a one-to-one basis to replace literally any of the 50 original files. It's been a long time but I used them extensively and it was a good way to correct errors or missing files. https://en.wikipedia.org/wiki/Parchive https://en.wikipedia.org/wiki/Parchive
    - deleted 7y ago
      [deleted]
    - cpach 7y ago
      Another weird thing is that in the early 10’s, people where still downloading warez over Usenet. Maybe it’s still a thing even today.
      - banana_giraffe 7y ago
        It still is. It's not quite the glory days of the alt.binaries.*, though. Now people share using nzb files, which means the actual posts on USENET are often just title-less binary blobs, impossible to know what they even are without the nzb files, which are often shared via private or semi-private websites or other channels.

by Isamu 7y ago

tptacek 7y ago

They ate up disk and CPU. It was the nature of NNTP, at least at the serious provider levels, that you had to be able to keep up in real time with a certain high level of responsiveness, or you'd miss your window for posts and there'd be holes in your binaries (because Usenet is perhaps the dumbest imaginable way to transmit a large binary, a Usenet binary was a chain of separate posts, all of which were needed to reassemble the binary). You could run a Usenet server without binaries, just to host the discussions and source code sharing and whatnot. But customers would have none of it: if you didn't have binaries, you weren't serving real Usenet, and they'd go to ISPs that did, and when ISPs generally stopped serving Usenet because it had become the world's lamest warez and porn distribution network, they moved to centralized NNTP services.

webmaven 7y ago
> (because Usenet is perhaps the dumbest imaginable way to transmit a large binary, a Usenet binary was a chain of separate posts, all of which were needed to reassemble the binary) I just realized that this is superficially similar to how bittorrent works...
- tptacek 7y ago
  Not at all. BitTorrent is forward error corrected. It splits binaries up into chunks as an optimization, and ensures that you don't need a precise sequence of chunks to reassemble the file; an error correcting code ensures that you only need k of n chunks. Usenet binaries were literally just raw binaries, uuencoded(!), and split up into n chunks, of which you need all n chunks. Those streams of chunks were then broadcast to every node on the network, despite the fact that only a subset of nodes ever wanted any particular binary. It was and remains the most batshit file transfer mechanism ever devised.
  - lvh 7y ago
    Not that I'd know: but at some point after the nineties Usenet figured out PAR files too, so you wouldn't quite need all the chunks because NNTP servers would drop some of them occasionally -- though to your point, yes, every NNTP server would de facto mirror everything.
  - 300bps 7y ago
    They actually fixed that problem around 2001! It's been over 15 years since I've done anything with Usenet binaries so I'm shocked I was able to recall the PAR file technology. Essentially, if you posted 50 parts of your encoded binary, you would generally post an additional 5 or 6 PAR files. Downloaders could then utilize a PAR file on a one-to-one basis to replace literally any of the 50 original files. It's been a long time but I used them extensively and it was a good way to correct errors or missing files. https://en.wikipedia.org/wiki/Parchive https://en.wikipedia.org/wiki/Parchive
  - deleted 7y ago
    [deleted]
  - cpach 7y ago
    Another weird thing is that in the early 10’s, people where still downloading warez over Usenet. Maybe it’s still a thing even today.
    - banana_giraffe 7y ago
      It still is. It's not quite the glory days of the alt.binaries.*, though. Now people share using nzb files, which means the actual posts on USENET are often just title-less binary blobs, impossible to know what they even are without the nzb files, which are often shared via private or semi-private websites or other channels.