5 ms·

Actually I think NNTP solved a bunch of problems that we didn't even know we had at the time, including scaling and funding. NNTP servers were generally provid

by Kadin 10y ago

Actually I think NNTP solved a bunch of problems that we didn't even know we had at the time, including scaling and funding. NNTP servers were generally provided by your ISP, so ISPs built the capacity out as they added subscribers, and added groups based on the perceived interest. It wasn't especially difficult (and it's pretty trivial today) to offer all the "official" text-based newsgroups with a few days or weeks worth of retention; it only becomes a significant undertaking when you want to offer months of retention for binaries newsgroups. Similarly, because the NNTP servers were basically a value-added ISP service, they were funded by subscriber fees. You never had a problem with a newsgroup suddenly getting popular and being unable to pay their own hosting fees, or suddenly tossing ads in order to pay the hosting bill, which is unfortunately common with webforums. And you don't have problems with a single server crashing and wiping out the history of a whole group. It's all pretty robust. Architecturally, netnews was superior to web forums. Where it failed was in not having a good response to the spam problem. I think that had it not been for spam, people wouldn't have left NNTP for HTTP-hosted webforums nearly as quickly. But the decentralized nature of netnews meant there wasn't anyone who could quickly implement any of the myriad solutions that were proposed (cryptographic proof-of-work plus reputation systems probably would have cut down on the worst offenders). My guess is that eventually, we'll reinvent something that's similar to NNTP but by another name (with the decentralization, server-server replication, gateways to other protocols, etc.), but it's been a long and unfortunate diversion in the meantime.

derefr 10y ago
I don't know about that: the core conceit—of ISPs doing any sort of edge server hosting—is nearly dead itself. Email is nearly-entirely owned by megacorps, unencrypted (and therefore edge-cacheable) HTTP is dying, DNS is basically just Google and OpenDNS at this point... soon ISPs won't run any OSI-level-7 services at all. They won't need data-centers, just switches. I don't see that trend reversing. And given that, you can't really have a store-and-forward edge-caching system that truly resembles NNTP. At most, you can get a hierarchical network like IRC.
- digi_owl 10y ago
  Pretty much. Around where i live, power companies have started offering fiber services. But they just pull the actual fiber along their existing power lines. Just about all of them has teamed up with a third party that handle not just ISP, but also TV and VOIP.
- dredmorbius 10y ago
  Of all of these, DNS is the easist to address -- anyone can set up their own forarding nameserver pointing at the roots. Performance is pretty good. Much as I distrust Google for anything these days, 8.8.8.8 / 8.8.4.4 offer a widespread advanced notice of domains tracking up the hit rates, and possible attacks. Email, as STMP, after decades of nobel service really simply does deserve to die. It's far less about the mechanics of email user interfaces, for which there's a great deal of ink wasted, and much more on privacy -- encryption, metadata, and spam, probably as the three top concerns. Very nearly all the suggested replacements (SMS, various messaging systems, Facebook) have either those problems or those and the fatal condition of being proprietary silos. The more I reflect on it the more the Web is close to what Usenet was, at least in some regards. Origins post messages, which may or may not die in short order (what's TTL for an Internet page now, a few months on average?). We're increasingly relying on sites such as the Internet Archive to actually provide a view-through-time at content. What the web (and email, especially in mailing lists) lacks and has always lacked is a requirement to meet standards conformance. DDG "The Web is an error condition" and you'll find a far-too-true-to-be-good rant on that topic. The loose-standards-anything-will-fly-nearly approach has allowed a great deal of flexibility in page design, but much of that is ultimately painful. I'm finding a much greater appreciation for straight-up HTML 1.0 formatted pages. A huge part of the problem is exceptionally crappy entity styling defaults. If browsers presented pages sanely (at various resolutions), much of the B-Arc profession of Web design would simply cease to exist. And this would be a wonderful thing. A simplified markup, acceptance criteria (and rejections for failed formats), cache-and-foreard, client-determined-formatting, security, and either a universal content syndication or UBI type compensation model might address other issues. I'm getting the sense of some interesting possiblities out there.
  - 10y ago
digi_owl 10y ago
I think a different issue was discoverability. With web forums you fire a search term into a random crawler and it will likely barf up a url. With NNTP you had to configure the email client to point to the isp server, and then locate a relevant group for what you were interested in. And even then searching the whole archive was troublesome if not impossible. I watched something similar play out with IRC. Once AJAX allowed random websites to offer chat services non-tech channels basically wasted away, as your random joe and jane just entered some url they were told, or read about in some glossy mag, and presto. As for reinventing NNTP, i don't think there is any incentive any longer. Especially now that something as big as Reddit can just pay Amazon to scale the back end as large as they need. Store and forward came to be before the net, and was used during the early net, because of narrow bandwidth.
wtbob 10y ago
> Actually I think NNTP solved a bunch of problems that we didn't even know we had at the time, including scaling and funding. Yeah, but think of the social scaling issues. You simply can't have every single person interested in e.g. American football reading & commenting on alt.games.football. News was indeed a far superior architecture & experience to web forums — so long as the userbase was relatively small.
- dredmorbius 10y ago
  I've shot around estimates of the size of Usenet with some old Usenetters (of whom I was one, though only marginally active). Gene "Spaff" Spafford thinks 50k - 500k users ~1988 - 1994 is probably ballpark accurate. Peter da Silva notes that in 1990 he was the most active user on Usenet ... and under another handle, the 3rd most active. Microsoft did some early 2000s research turning up low-single-digit million distinct user IDs IIRC (I've posted on this previously). So, yeah, it was small. Organising people into groups, even ad-hoc, from which exceptionally good messages might be shared to others, might be one way of handling scaling. Clustersizes ranging from a total size within Dunbar ranges, to a 1/9/90 split giving the 10% active users a Dunbar cap (about 300 - 3,000 users per cluster) might work. I agree that 1990 Usenet in 2016 would be a fatal mistake.
- rakoo 10y ago
  How could alt.games.football not scale, and /r/football scale ? I wouldn't expect a single server, or even a single network host the whole conversation about football in the world. So you'd have reddit.com's alt.games.football, facebook's alt.games.football and so on. Or, more probably, brand new hierarchies like com.facebook.games.football and com.reddit.games.football. The nice thing is that they'd all have the same interface and the same protocol so they are both easily accessible and easily archivable.
  - rmc 10y ago
    > How could alt.games.football not scale, and /r/football scale ? Upvoting. NNTP didn't have it. Reddit does. This allows a reader to see the top posts.
gwbrooke 10y ago
Well managed servers don't have spam. I used http://news.individual.net/ http://news.individual.net/ for a while and I don't recall getting any spam, ever.

by Kadin 10y ago

derefr 10y ago

I don't know about that: the core conceit—of ISPs doing any sort of edge server hosting—is nearly dead itself. Email is nearly-entirely owned by megacorps, unencrypted (and therefore edge-cacheable) HTTP is dying, DNS is basically just Google and OpenDNS at this point... soon ISPs won't run any OSI-level-7 services at all. They won't need data-centers, just switches. I don't see that trend reversing. And given that, you can't really have a store-and-forward edge-caching system that truly resembles NNTP. At most, you can get a hierarchical network like IRC.

digi_owl 10y ago
Pretty much. Around where i live, power companies have started offering fiber services. But they just pull the actual fiber along their existing power lines. Just about all of them has teamed up with a third party that handle not just ISP, but also TV and VOIP.
dredmorbius 10y ago
Of all of these, DNS is the easist to address -- anyone can set up their own forarding nameserver pointing at the roots. Performance is pretty good. Much as I distrust Google for anything these days, 8.8.8.8 / 8.8.4.4 offer a widespread advanced notice of domains tracking up the hit rates, and possible attacks. Email, as STMP, after decades of nobel service really simply does deserve to die. It's far less about the mechanics of email user interfaces, for which there's a great deal of ink wasted, and much more on privacy -- encryption, metadata, and spam, probably as the three top concerns. Very nearly all the suggested replacements (SMS, various messaging systems, Facebook) have either those problems or those and the fatal condition of being proprietary silos. The more I reflect on it the more the Web is close to what Usenet was, at least in some regards. Origins post messages, which may or may not die in short order (what's TTL for an Internet page now, a few months on average?). We're increasingly relying on sites such as the Internet Archive to actually provide a view-through-time at content. What the web (and email, especially in mailing lists) lacks and has always lacked is a requirement to meet standards conformance. DDG "The Web is an error condition" and you'll find a far-too-true-to-be-good rant on that topic. The loose-standards-anything-will-fly-nearly approach has allowed a great deal of flexibility in page design, but much of that is ultimately painful. I'm finding a much greater appreciation for straight-up HTML 1.0 formatted pages. A huge part of the problem is exceptionally crappy entity styling defaults. If browsers presented pages sanely (at various resolutions), much of the B-Arc profession of Web design would simply cease to exist. And this would be a wonderful thing. A simplified markup, acceptance criteria (and rejections for failed formats), cache-and-foreard, client-determined-formatting, security, and either a universal content syndication or UBI type compensation model might address other issues. I'm getting the sense of some interesting possiblities out there.
- 10y ago

digi_owl 10y ago

I think a different issue was discoverability. With web forums you fire a search term into a random crawler and it will likely barf up a url. With NNTP you had to configure the email client to point to the isp server, and then locate a relevant group for what you were interested in. And even then searching the whole archive was troublesome if not impossible. I watched something similar play out with IRC. Once AJAX allowed random websites to offer chat services non-tech channels basically wasted away, as your random joe and jane just entered some url they were told, or read about in some glossy mag, and presto. As for reinventing NNTP, i don't think there is any incentive any longer. Especially now that something as big as Reddit can just pay Amazon to scale the back end as large as they need. Store and forward came to be before the net, and was used during the early net, because of narrow bandwidth.

wtbob 10y ago

> Actually I think NNTP solved a bunch of problems that we didn't even know we had at the time, including scaling and funding. Yeah, but think of the social scaling issues. You simply can't have every single person interested in e.g. American football reading & commenting on alt.games.football. News was indeed a far superior architecture & experience to web forums — so long as the userbase was relatively small.

dredmorbius 10y ago
I've shot around estimates of the size of Usenet with some old Usenetters (of whom I was one, though only marginally active). Gene "Spaff" Spafford thinks 50k - 500k users ~1988 - 1994 is probably ballpark accurate. Peter da Silva notes that in 1990 he was the most active user on Usenet ... and under another handle, the 3rd most active. Microsoft did some early 2000s research turning up low-single-digit million distinct user IDs IIRC (I've posted on this previously). So, yeah, it was small. Organising people into groups, even ad-hoc, from which exceptionally good messages might be shared to others, might be one way of handling scaling. Clustersizes ranging from a total size within Dunbar ranges, to a 1/9/90 split giving the 10% active users a Dunbar cap (about 300 - 3,000 users per cluster) might work. I agree that 1990 Usenet in 2016 would be a fatal mistake.
rakoo 10y ago
How could alt.games.football not scale, and /r/football scale ? I wouldn't expect a single server, or even a single network host the whole conversation about football in the world. So you'd have reddit.com's alt.games.football, facebook's alt.games.football and so on. Or, more probably, brand new hierarchies like com.facebook.games.football and com.reddit.games.football. The nice thing is that they'd all have the same interface and the same protocol so they are both easily accessible and easily archivable.
- rmc 10y ago
  > How could alt.games.football not scale, and /r/football scale ? Upvoting. NNTP didn't have it. Reddit does. This allows a reader to see the top posts.

gwbrooke 10y ago

Well managed servers don't have spam. I used http://news.individual.net/ http://news.individual.net/ for a while and I don't recall getting any spam, ever.