7 ms·
Can someone explain Usenet to me? As I understand it, it's not a website. Is it something outside of the WWW? Does it have its own protocol or does it use HTTP?
by bonoboTP 6y ago
Can someone explain Usenet to me? As I understand it, it's not a website. Is it something outside of the WWW? Does it have its own protocol or does it use HTTP? Do you need a dedicated client software for it, similar to IRC? How do/did you register to it? Is it paid? Who hosts it?
- dylan604 6y agohttps://en.wikipedia.org/wiki/Usenet https://en.wikipedia.org/wiki/Usenet
- not2b 6y agoIt was the world's biggest distributed BBS system. Originally, Usenet used UUCP for transport; later on NNTP was defined so Usenet could be shared via TCP/IP. In the early days, only DARPA contractors (which included many university computer science departments) could be on the Internet; second-class citizens could use UUCP and modems.
- johannes1234321 6y agoAn important part about usegroups is that it is from an era of dial-in connections. One can dial into your ISP, sync News (fetch new messages in groups one is subscribed to and send messages) and then go offline and read/answer the messages without paying per minute and without blocking ports for other dial-in customers. This also worked between different news servers in distinct networks with unreliable connections. For the how paid: In some regions it was part of the things a "good" ISP had to offer their clients. Also back in the days™ large parts of the internet was run on machines in some university labs.
- bonoboTP 6y agoI see. It seems somewhat like mailing lists to me. I go online, my client sends and receives messages depending on which groups I'm subscribed to, and messages bounce around in a distributed fashion, somewhat like emails bouncing between SMTP servers. I still don't quite get the user experience. What if you're subscribed to alt.binaries? Do you then sync all the binaries when you go online? That wouldn't make sense. Perhaps it only synchronizes on a level below that, alt.binaries.something.something? But that may still be too much. And what if my ISP's server didn't have the newsgroup I was interested in? Is it like DNS, where it would then fetch it from somewhere else and then cache it? Or did most servers store the entire data? Or an ISP would just store the messages that at least one of their customers subscribe to?
- johannes1234321 6y agoYes, it is related to mail, but with open groups. Country to mailing list you have the complete archive. In a way a world readable impact server with append permissions (well, actually there are ways to pull old messages back etc.) and synchronisation between servers. (Not correct comparison, but maybe helps) The individual messages are also structured like mail using MIME. For what you download in sync depends on your config. alt.binaries is special, I believe relatively young and many providers didn't offer them to begin with ... When fetching you configured your client "please fetch all of comp.lang.cpp" and it would do. I believe you could also say "just give me the headers" and load only selected messages on next sync (i.e. for fetching only binaries from messages with interesting subjects) If your ISP didn't have a server, there were multiple open ones in universities or other ISPs (not all locked down for their customers only but either didn't care or saw it as an ad for them), but they could be far away (modern cross ocean cables are younger ...) and busy/slow. And yes, a server can decide which groups it syncs.
- jdeibele 6y agoYou would (typically) have a dumb terminal experience. You'd use your computer and modem to log into a remote system. You would use a newsreader to browse newsgroups. If there was something that you wanted, like binaries, they would be saved on the remote system. You'd then need to ftp them from that system to your own. If you were one of the fortunate ones to have a hard-wired connection, well, that happened a lot faster. Newsreaders were awesome for reading newsgroups. You were able to add someone or a topic to a kill file for one group or all groups. As binaries got bigger and bigger, the retention time for them on the remote system got shorter and shorter. You might have 7 days of binaries and 90 days of everything else. For a while some forum software would be accessible via NNTP which was awesome. It let more people use the forums but gave more experienced users much better tools.
- gwittel 6y agoSort of like mailing lists with a dedicated client so you can see everything in nice threaded fashion. You could explore new groups through the client as well. I never found the web based Usenet mirrors comparable to a dedicated client. Typically you synced the equivalent of headers and bodies separately (for the groups you were interested in). This way you only grabbed what you wanted. For binary groups, people settled on patterns in subjects, etc so that clients could automate downloading of files (binaries had to be encoded and split into parts due to post length limits). On regular text type newsgroups you might set your client to download headers+bodies in one shot for offline mode. With offline mode you would then just download what's new vs your cache. Then work against your local cache for reading/replying. Once online again, your batched replies could be sent. If your ISP didn't have the newsgroup you wanted, you would have to point to another NTTP server. For a while there were public ones that anyone could use. Over time, those public ones went away or dropped problematic areas like alt.binaries.*. These days Usenet is available as a for-pay service.
- retrac 6y agoYes. It's outside the WWW. Originally, it was outside the Internet. The basic concept is simply copying and forwarding email-like messages, grouped into a hierarchical set of newsgroup feeds. A host would subscribe to other hosts, and would allow other hosts to subscribe to their own feeds. And then just push/pull this through the network. Originally, this was done all with shell scripts dialing up remote hosts by modem and copying compressed datasets nightly. This developed into a full suite of protocols called UUCP. Later, Internet protocols like the Network News Transfer Protocol started carrying some, then all, of the traffic. In principle, it is unregistered, unpaid, and decentralized. You start exchanging with someone else who has the newsgroups you want to send and receive. In practice, both in the past and now, you subscribe, usually commercially, to one of the services with the bandwidth and storage to host a reasonable portion of Usenet. ISPs often provided NNTP Usenet service into the early 2000s, along with SMTP/POP for email.
- bonoboTP 6y agoSo basically there was a fully synced copy of the entire newsgroup contents for the groups I'm subscribed to? Like it would fetch all the new messages that arrived in alt.atheism and store it on my disk? But that surely wouldn't work on alt.binaries. Was there something like a cutoff at a certain size and you'd have to explicitly request the full content for some specific messages?
- aeyes 6y agoUsing the NNTP protocol (which is very similar to SMTP) each message has two parts: Headers and content. Your client would usually only download message headers which contains information like author, date, title and message size. Back when UUCP was still around there wasn't much binary data in groups. It took a while to develop algorithms to encode binary data (uuencode, base64), the protocol was text only.
- throwaway2048 6y agogenerally speaking only usenet servers themselves grab everything at all times and store it, subject to an retention time (say 1 year, messages older than this are deleted) most clients just grab headers, and download messages on access.
- irrational 6y agoNot only was it outside of the WWW, it predated the existence of the WWW by a good 15 years.
- indymike 6y agoOne of the more interesting things about usenet was it was all text, and when the web took off, some readers (ahem Netscape) tried to push HTML messages into Usenet. Btw... Usenet was incredible for in-company communication. Used to have a private newserver for our employees with names like .customer.help .hr.questions and so on. Was super useful until someone decided to try to put all the things in Outlook.
- wazoox 6y agoAh yes, we had an NNTP server as a dev and support chat/group messaging in my startup in 1999-2002 :) It was great: searchable, grepable, all discussions neatly threaded, groups by subjects...
- webmaven 6y ago> Btw... Usenet was incredible for in-company communication. Used to have a private newserver for our employees with names like .customer.help .hr.questions and so on. Was super useful until someone decided to try to put all the things in Outlook. Jon Udell's 1999 book "Practical Internet Groupware" was an excellent resource on how to set up this sort of infrastructure by gluing together various protocols: https://g.co/kgs/ExEfcX https://g.co/kgs/ExEfcX
- jrm4 6y agoThink something like Reddit + Bittorrent. Slight difference -- there would be no main "front page," -- only individual self-hosted hubs that could choose which "subreddits" (newsgroups) they would carry.
- bonoboTP 6y agoHow did you get a username? What was its format? Something structured like an email address or just any string?
- jrm4 6y agoI'd say that's where my analogy breaks down a bit. Reddit has a fully centralized username system, in Usenet it would be up to the individual hub (as I recall?) The Mastodon ecosystem today is a better analogy here.
- ars 6y agoIt was normally your email address, but you could put anything. Typically the posting news server (i.e. your own news server) was responsible for validating that you used your real email address. Originally there was no validation, but as spam rose validation became required. News servers required you to login, originally they were open to anyone.
- indymike 6y agoYou just made something up. If you wanted to ensure your messages were untampered, you signed with PGP.
- toyg 6y ago“Reddit + bittorrent” would actually be pretty cool...
- deleted 6y ago[deleted]
- thesuitonym 6y ago
- basch 6y agoin overly simple terms http+html+url=www, born in 1989.
- visiblink 6y agoGo to eternal-september.org, sign up, set up Thunderbird or another NNTP client, and you can experience Usenet today in all its glory. Comp.misc is a good group to start with.
- bonoboTP 6y agoThanks! I just did it. Very interesting.
- tzs 6y agoIf you are comfortable with text-based interfaces, I'd suggested getting slrn to see what Usenet was like back in the days when most of us only had text terminals. If you are an Emacs user, Gnus is another way to get the old school experience. Another text one that was popular is tin, which I've never used but which, like slrn, is still maintained and updated.
- visiblink 6y agoIf you want to set up slrn -- as suggested in another post -- I can help. It's not particularly intuitive. I first used Claws Mail and Thunderbird and then set up slrn. I still use Thunderbird most often.
- pmachinery 6y agoThe above post is the best answer to what is Usenet; understand it from using it. For anyone who doesn't want to sign up to try it, there's also: news.aioe.org https://www.aioe.org https://www.aioe.org
- ChuckMcM 6y agoHad its own protocol, "NetNews", and used a peer to peer network called "UUCP" which is short for "unix to Unix copy" (the unix copy command being cp). One would "post" a message and periodically one or more systems would poll your system. Either over the ARPAnet or via an actual modem. It would fetch all of the posts that had been created on that system and add them to its list of new posts, and it would send you all of the posts you hadn't yet seen. In a directory on your system would be all of the posts from everyone on Usenet for all of the groups you cared to "follow." Back in the 80's I could read everything posted in a couple of hours in the morning. :-) That changed quickly.
- chadcmulligan 6y agoI worked at a uni that had usenet, soon after graduating. There were newsgroups about everything, personally most mornings at work I'd look at comp.lang.c and others it was a great resource for learning - it was one place and all the uni's in the world were connected to it. I learnt so much from it. Messages were transferred overnight by copying the latest changes between each uni, you could post to the use group of choice using a usenet client. I was very excited to have access to it at the time, it was rare to have a connection outside of a uni. There were source code feeds to where you could download the latest sources and try to compile them yourself, I remember fiddling with things like flex and bison. It was full nerd - pretty well everyone was an academic, student or worked at a uni or research lab, you'd have contact in almost real time with anyone - this was a new thing. You'd interface with it using a glass tty - I had a vt52 which connected to a Sun 3 box, which was our development computer. I remember getting a 300 baud modem and connecting to it at night from home with my PC, or maybe it was my amstrad still, PC's were very expensive still. It was a fun time. Edit: just remembered we only had one dial in modem to the uni, and I'd use it to dial in from home, I remember we used to fight over access to it - eventually they got a second one because the sys admin was sick of not being able to get access to the network at night. They to were very expensive, I think they were 2400 or maybe even 4800 bps - a crazy speed then.
- tzs 6y agoOthers have covered what Usenet was. To further give you an idea of how different those times were, here is my email address from the signature on one of my Usenet posts from the '80s: ihnp4!{cithep,wlbr!callan}!tim What that meant was that to get an email to me, your computer should arrange to deliver it to ihnp4. That was the name of a machine at a Bell Labs office at a place named "Indian Hill" in Naperville, IL. It stood for "Indian Hill Network Processor 4". ihnp4 was very well connected, so it was reasonable for me to expect that you could find a way to get the email to it. The rest of my email address is saying that ihnp4 should deliver the mail to cithep (Caltech High Energy Physics) or wlbr (a machine at the Westlake Village, CA offices of Bunker Ramo). If it goes to wlbr, wlbr should send it to callan (a machine at Callan Data Systems). I had accounts on cithep and callan, both with user name tim. You didn't have to start at ihnp4, of course. As long as you could find a way to get it to one of {ihnp4, cithep, wlbr, callan} you could get it to me. Periodically there was a posting on Usenet giving a list of machines that would publicly relay mail and their interconnections. (I want to say this was monthly and in net.general, but I'm not sure). Given that and a list of machines that your site connected to, you could find a path from your site to me. Let's say you were in Boston, and you knew your company exchanged mail with decvax (a machine at Digital Equipment Corporation). You could see from the connection list that decvax connected to trwrb (a machine at TRW Redondo Beach, CA, I believe), and trwrb connected to wlbr, so from your machine the email address decvax!trwrb!wlbr!callan!tim would work to reach me.
- SiVal 6y agoRe: "different times", VAX sysadmins in those days of "bang-bang" addresses were like telegraph offices. We didn't yet have what we called "glass teletypes", so we would "post" to a newsgroup by writing the post on paper and posting it in the sysadmin's inbox (an actual box). He would type it in sometime during the day, and our leased-line connection would exchange messages with MIT several times a day. Each morning, the sysadmin would print out the previous day's postings in each of the several newsgroups we followed. One stack of fan-fold, green-striped lineprinter paper for each newsgroup that we cared about (maybe 4-6). During the day, we would pass the "newsgroup" around the office like a daily newspaper.
- 6y ago
- dboreham 6y agoPeer to peer discussion groups.
- peterwwillis 6y agoOne of the things it eventually became was the world's largest, fastest and most reliable media-pirating service, paid for and hosted by your ISP. Eventually the binary groups got dropped by ISPs, but you could pay to use someone else's servers and pull a couple hundred gigabytes of content at fiber speeds. This all while people were struggling for weeks to get a single torrent to finish. Of course, FXP servers and IRC D/C were the OG pirating method, but usenet binaries were accessible to pretty much anybody and in no way illegal ("I'm just downloading the news!"). Though it also seemed like nobody even noticed anything illegal was going on, because most people were stuck on P2P alternatives even though they sucked in comparison. I'm pretty sure Napster succeeded because normal people couldn't comprehend Usenet.
- peter303 6y agoBefore there was the internet, many UNIX computers communicated by calling each other on telephone modems a few times a day. Among the exchanged texts were email and the discussion blog called usenet. In the early there were only a few dozen to few hundred computers, mainly in universities and computer companies. You list all the networked computers in a single file called /etc/hosts. In the late 1980s universal protocols and funding from Senator Al Gores 1991 Information Superhighway legislation allowed the computers anywhere in the world to interconnect. My usenet email address from 1976 still works.
- anakaine 6y agoA 44 year old email address. Thats impressive.
- dredmorbius 6y agoFor those interested in early online systems, I strongly recommend John S. Quarterman's The Matrix: Computer Networks and Conferencing Systems Worldwide: https://archive.org/details/matrixcomputerne0000quar/mode/2up https://archive.org/details/matrixcomputerne0000quar/mode/2u... I lifted a copy off a friend who was thinning his book herd a decade or so back. The first edition, published in 1990, predates the commercial Internet, World Wide Web, Linux, and all of what's transpired since, though the introduction pressages much by noting the role of existing networks in the the then-fresh events of April 15 – June 4, 1989, known as the six-four incident, and culminating in the Tiananmen Square Massacre. The book has a 15 page section describing Usenet, including user and traffic statistics from 1979--1988, mostly compiled by Gene Spafford and Brian Reid. Usenet is only one of numerous conferencing systems covered, others being BITNET, UUCP, FidoNET, VNET, and EASYnet For April, 1988: 381 newsgroups, 57,979 messages, 130.949 MB, 4,152--7,810 hosts (depending on who counted and how), and an estimated 141,000 readers (people) from a total online population of 880,000. Larry Wall, working at JPL, is named ... as the author of rn. Much of the book, though, simply discusses the many many individual computer networks. The notion of a single global end-to-end interconnected inter-network was still very much a novelty. In one section, the fact that the principle public interconnect between North America and Europe was a single 9600 Baud link is mentioned....
- deleted 6y ago[deleted]
- mcv 6y agoUsenet indeed predates the Web and HTTP, and while it mostly existed on the internet, it could also propagate to machines not connected to it, or connected through dialup lines, using uucp (unix-to-unix-copy). What it was is basically a massive collection of discussion groups, organised hierarchically, hosted on local machines that exchanged these messages with other machines. There was a variety of newsreader programs available that you could use to read and post messages on these discussion groups. Not every usenet host supports every group. There were a couple of top-level hierarchies, like rec.* for everything recreational, like hobbies and games (I was an avid reader of the rec.games.frp.* newsgroups, which were about roleplaying games); sci.* for everything science, comp.* for everything computer related (some of which was admittedly also recreational). There was also the alt.* hierarchy where literally anything goes. Many hosts only supported a limited selection of the alt.* newsgroups. You'd post under your own email address, and most newsreaders also supported emailing people directly instead of responding in the group. I believe there was a big revamp at the end of the 1980s where everything about usenet got changed. I only know the situation from after that reorganisation.
- ape4 6y agoFrom the user's point of view there was a Unix command `rn` for Read News. Could also post.
- throw0101a 6y agoUsenet was a peer-to-peer 'forum' system that used the NNTP protocol. An NNTP server had a list of discussion forums: clients connected and got the list, and the user could selectively subscribe to the one's they were interested in. The client would keep track of which messages were read, and so when new ones were posted you'd only see new messages by default. Each NNTP server could talk to other NNTP servers. They would exchange a list of forums each had, and then the articles had in each forum. Each article/post had a unique ID (basically like the Message-ID in e-mails), and so each could request articles from the other that they did not already have. New messages could be posted to either server and they would 'cross pollinate' (multi-master basically). Each NNTP server could technically talk to multiple other NNTP servers, so cross-pollination could be very wide, but in the early days with low-kbps modems and expensive long distance, there was a lot of local hierarchical hubs. There were a few key hubs that had fancy long-distance T1 and such that could transfer across continents and oceans as a bit of a service for the community.