2 ms·

So basically there was a fully synced copy of the entire newsgroup contents for the groups I'm subscribed to? Like it would fetch all the new messages that arri

So basically there was a fully synced copy of the entire newsgroup contents for the groups I'm subscribed to? Like it would fetch all the new messages that arrived in alt.atheism and store it on my disk? But that surely wouldn't work on alt.binaries. Was there something like a cutoff at a certain size and you'd have to explicitly request the full content for some specific messages?

aeyes 6y ago
Using the NNTP protocol (which is very similar to SMTP) each message has two parts: Headers and content. Your client would usually only download message headers which contains information like author, date, title and message size. Back when UUCP was still around there wasn't much binary data in groups. It took a while to develop algorithms to encode binary data (uuencode, base64), the protocol was text only.
throwaway2048 6y ago
generally speaking only usenet servers themselves grab everything at all times and store it, subject to an retention time (say 1 year, messages older than this are deleted) most clients just grab headers, and download messages on access.
dudleypippin 6y ago
I think the key bit is where "my" is in that second sentence. At the time, the main server (or your hosting service) would, indeed, have the full list of messages for all the groups they carried, with a lookback of a certain number of days (the "retention"). The retention and the groups that a hosting service subscribed to would be part of how you would pick which to sign up with. Your disk, however, would only have the messages you'd specifically requested and pulled down over your modem. (Or, if you went way back to the vt100 days, "your disk" was the central server which had it all.)
jcranmer 6y ago
Most NNTP servers wouldn't store the whole archive. Usually, they'd retain something like 3 months of archives. Providers actually competed on how much archival history they retained. Retention also usually varies based on binary/text newsgroups: there is far less cost to retaining text Usenet posts because they're usually very small. My 1M message Usenet archive is 5.6GiB of space, and I believe this covers ~30 days of non-alt.binaries messages. The data is a decade old, so I don't recall the exact details of collection anymore.
ilkkao 6y ago
I had a summer job in 2001 which included helping the newsmaster at one of the biggest ISP at the time in Finland. If I remember correctly, there was a dedicated satellite dish to get the full newsfeed from some specialised newsfeed operator. Binary groups were popular then and this was done to avoid the huge international traffic via the normal routes.

aeyes 6y ago

Using the NNTP protocol (which is very similar to SMTP) each message has two parts: Headers and content. Your client would usually only download message headers which contains information like author, date, title and message size. Back when UUCP was still around there wasn't much binary data in groups. It took a while to develop algorithms to encode binary data (uuencode, base64), the protocol was text only.

throwaway2048 6y ago

generally speaking only usenet servers themselves grab everything at all times and store it, subject to an retention time (say 1 year, messages older than this are deleted) most clients just grab headers, and download messages on access.

dudleypippin 6y ago

I think the key bit is where "my" is in that second sentence. At the time, the main server (or your hosting service) would, indeed, have the full list of messages for all the groups they carried, with a lookback of a certain number of days (the "retention"). The retention and the groups that a hosting service subscribed to would be part of how you would pick which to sign up with. Your disk, however, would only have the messages you'd specifically requested and pulled down over your modem. (Or, if you went way back to the vt100 days, "your disk" was the central server which had it all.)

jcranmer 6y ago

Most NNTP servers wouldn't store the whole archive. Usually, they'd retain something like 3 months of archives. Providers actually competed on how much archival history they retained. Retention also usually varies based on binary/text newsgroups: there is far less cost to retaining text Usenet posts because they're usually very small. My 1M message Usenet archive is 5.6GiB of space, and I believe this covers ~30 days of non-alt.binaries messages. The data is a decade old, so I don't recall the exact details of collection anymore.

ilkkao 6y ago

I had a summer job in 2001 which included helping the newsmaster at one of the biggest ISP at the time in Finland. If I remember correctly, there was a dedicated satellite dish to get the full newsfeed from some specialised newsfeed operator. Binary groups were popular then and this was done to avoid the huge international traffic via the normal routes.