12 ms·

This, along with recent Reddit goings-on has made me realize a major risk with the current structure of online communication. Take either Reddit or Stack Exchan

by lumb63 3y ago

This, along with recent Reddit goings-on has made me realize a major risk with the current structure of online communication. Take either Reddit or Stack Exchange as examples. They build a platform, and users contribute their time, thought, energy, and knowledge to build a community on that platform. Those companies can then gatekeep and restrict access to all that the community built, when all they did is provide the platform, and store the data. We need to rethink this model. The thought and knowledge of communities and users need to belong to those communities and users. To people they intentionally and thoughtfully delegate to and trust. We need to decentralize our communications, like how the internet used to be before the arrival of social media and mega forums. We need to revert to small, focused forums, with less anonymous, more persistent communication, run by people we trust. Otherwise, we will continue to see mega companies harvest our data and use it (or not provide it) against our wishes. If we don’t work to mitigate that dynamic, we have nobody to blame for the poor outcomes but ourselves.

kiba 3y ago
Perhaps these sort of things shouldn't be for profit enterprise, given the inability of companies to not slaughter the goose that lay the golden eggs.
- phailhaus 3y ago
  The problem is defining "these sorts of things". StackOverflow didn't do anything evil, they created a useful website and people flocked to it voluntarily.
the_pwner224 3y ago
A decentralized system will never work because 99% of users do not care at all; the centralized systems are easier to sign up for and use. It's been demonstrated over and over and over again. Even if the underlying tech is decentralized, the community will settle around one or a few big instances (for example, Gmail and GitHub) which often end up having significant control over the trajectory of the entire ecosystem. If you run your own email server and you get put onto Google's spam list - you're fucked.
- tedivm 3y ago
  >If you run your own email server and you get put onto Google's spam list - you're fucked. It's even worse than that- I ran my own email server, and for some reason gmail delayed any emails from their system to outside of their system. That meant that people would send me an email but I wouldn't get it for 20 minutes. These delays don't exist when using big email providers (it stopped being a problem when I switched to Fastmail, for example) but if you're running a small server Google makes it a nightmare.
  - OfSanguineFire 3y ago
    I have run my own email server off a Linode for the last decade-plus, and I have never encountered this. Most of the people with whom I correspond (I run my own business from my server) are on Gmail, and I have always received their emails instantly. If you were getting emails only 20 minutes later, I wonder if there was some server misconfiguration on your end, e.g. sending messages into graylisting delays.
  - dataflow 3y ago
    I assume you didn't have anything in between Gmail and your setup? Like a forwarding layer or something?
  - jethro_tell 3y ago
    Sounds like you need to allowlist google from your grey list. They have a long retry on their side. Once you tell a server to 'go away, come back later' it is really up to the server to decide when or if to retry. Additionally, If they use multiple sending IPs, you can end up grey listing again and again before they try back with a good ip. You'd either need to allowlist the big providers sending blocks or just drop grey listing all together.
    - actionfromafar 3y ago
      My gray list behaves like this. It's seldom a problem in practice, but new senders may need a "warm-up".
xen2xen1 3y ago
The risk of centralized systems was discussed long ago. The Cathedral and the Bazarr was published in 1999. None of these ideas are new. Everyone who payed any attention knew it was coming.
jacquesm 3y ago
Ever since Gracenote/CDDB it was pretty clear that this is the model. Still pissed off about that.
ethanbond 3y ago
Really? The takeaway isn’t rather that rent-seeking AI models need to figure out a way to reimburse companies and communities who’ve stored up all this capital? Seems to me SO built and delivered huge, huge amounts of value and it’s now all at risk because multibillion dollar companies are free riding.
- ImPostingOnHN 3y ago
  SO delivered some value, the users are the ones who delivered huge, huge amounts of value
  - ethanbond 3y ago
    Right, which SO (and not some other site) managed to entice. Somehow users didn’t flock to my ethansusefulprogrammerquestionandanswers.com ¯\_(ツ)_/¯
    - ImPostingOnHN 3y ago
      It seems more likely that there, too, it was the users doing the work, not SO. the primary value on SO is generated by the users, and thus the value proposition to enticing new users is also generated by the users. SO is just a forum.
  - croes 3y ago
    And companies like OpenAI will take the profit and even kill some of the users jobs.
    - ImPostingOnHN 3y ago
      and the heat death of the universe will eventually extinguish all life
      - croes 3y ago
        I think these are different time frames
dylan604 3y ago
How did newsgroups work in the days of yore? Depending on which ISP you used, you may or may not have all of the posts within a group if they even had the group. I remember paying for access to specific (can't remember the name) provider that had the most complete listing of newsgroups and had the longest retention of posts. viva a.b.m.a!!
- shagie 3y ago
  https://en.wikipedia.org/wiki/Network_News_Transfer_Protocol https://en.wikipedia.org/wiki/Network_News_Transfer_Protocol Originally over UUCP (Unix Unix Copy Protocol) and done via dial ups at night (when the rest of the batch transfers were done - email too with the old bang path). The two servers would exchange all the batched email and news posts that were routed to the other side. RFC 977 ( https://www.w3.org/Protocols/rfc977/rfc977 https://www.w3.org/Protocols/rfc977/rfc977 ) has an example of how files are copied between the two systems (section 4.6) including fetching and receiving mail. Note that not all posts outbound are necessarily of interest to the other server. An IHAVE message could come back with either a "I want it" type response or a "not interested" > The IHAVE command informs the server that the client has an article whose id is <messageid>. If the server desires a copy of that article, it will return a response instructing the client to send the entire article. If the server does not want the article (if, for example, the server already has a copy of it), a response indicating that the article is not wanted will be returned. That's how some of the moderation worked - your server would say "I don't want anything that came by way of X host" or "not interested in that newsgroup." One of the amusing things to me (looking back at this), if you're familiar with HTTP response codes, you'll likely get most of the way through the NNTP ones. 200 server ready - posting allowed 400 service discontinued 411 no such news group 500 command not recognized I'd also suggest a read of RFC 850 ( https://www.w3.org/Protocols/rfc850/rfc850.html https://www.w3.org/Protocols/rfc850/rfc850.html ) for some other background and section 5: The News Propagation Algorithm
  - dylan604 3y ago
    But how did all of this line up within the federated or not conversation? If each ISP could host their own version, that doesn't sound federated. But who was in control of the "main" source of truth type of version?
b3morales 3y ago
This was one of the promises originally of Stack Overflow: all the content is Creative Commons licensed so that if they "turned evil" (I believe it was Joel that put it this way) the community could, in a way, create a fork. https://web.archive.org/web/20230203170609/https://stackoverflow.blog/2009/06/04/stack-overflow-creative-commons-data-dump/ https://web.archive.org/web/20230203170609/https://stackover... Unfortunately the dumps themselves are not a legal requirement, just a gentleman's agreement, so realistically exercising this ability was still at the whim of the company.
- wahnfrieden 3y ago
  So it's time to community fork?
  - b3morales 3y ago
    Assuming that the linked post is accurate and that the "approval from senior leadership" to turn the dump back on does not come...then yes, I would say so. Actually there is already Codidact, although if I recall correctly they explicitly ruled out importing SE data when they started up. https://codidact.org https://codidact.org
    - deleted 3y ago
      [deleted]
    - Wowfunhappy 3y ago
      Yeah, Codidact isn't a "fork" because they don't use the SE data.
      - shagie 3y ago
        It’s a fork of the community rather than the data and content.
        sitkack 3y ago
scarface_74 3y ago
> when all they did is provide the platform, and store the data And all Google did was build a search engine.
nightski 3y ago
I like the idea of decentralized but I'd suggest you don't need to go fully decentralized where every peer has a full copy. Actually I kind of like the Bitcoin approach in that you have the ability to create a full peer, but most people do not. This would allow some decentralization and reduce risk, but not burden everybody with running a full peer.
jupp0r 3y ago
Incentives in this structure go both ways though, ideally keeping everything in symbiotic balance. Companies that alienate their users tend to not do well shortly after.
ilyt 3y ago
That doesn't stop grumpy admin of a federated instance to just do exactly same thing. > run by people we trust People change, or retire, just like corporation goals change. Focusing on more independent is not enough. If you want truly unbreakable stuff first part of the puzzle is saving user's handle and identity in a way that can't be removed. Then finding out a way to link that to their content so when place of hosting it goes away people can follow to the new place Then just have all of that content be signed by that identity so users can verify that it is really that person. And I can't believe I'm saying that unironically but blockchain might just be the solution for that. Something like immutable log of: * user declaring "I'm jeff@example.com, here are my public keys". Servers then validate via DNS record or some .well-known location entry whether user is allowed to declare they are from @example.com * user declaring "behold! jeff@example.com stuff is <here>, and <here> and <here> are addresses for various federation systems". Only passes if that request is signed with above privkey of course * user declaring "behold! My new public key is X and Y. And Z key is revoked!" * user declaring "behold! I am now george.effluent@company.com! Re-does checks but for new domain and users previously subscribed to jeff@example com get served redirect". etc. Then when server admin inevitably goes rogue you can take your posts and subscribers and go somewhere else. And when @example.com owner decides "well I'm just gonna to redirect stuff to ads", you can just change your handle and direct people to right place, and other handle is forever taken.
deleted 3y ago
[deleted]
deleted 3y ago
[deleted]
einpoklum 3y ago
> The thought and knowledge of communities and users need to belong to those communities and users. I would say: Need to be a public resource, belonging to no-one, i.e. no person, or group, or company should have legitimacy in denying access to it. They should all be considered _trustees_ of such a resource. > when all they did is provide the platform, and store the data. To be fair, SE Inc. did a lot more than provide the platform. A lot of development and design work, publication, a bit of the curation work, etc. I don't like how they behave but let's give them what they're due. --- Also note the ongoing Moderator Strike (!): https://meta.stackexchange.com/q/389811/196834 https://meta.stackexchange.com/q/389811/196834
briffle 3y ago
I'd argue its not just forums, but other key parts of the internet. Like Microsoft training co-pilot AI on github code, but not following the licensing of some code they straight up copy and suggest. I'm kind of curious what is next.
erlend_sh 3y ago
Agreed, and working on it: https://blog.erlend.sh/assembling-community-os https://blog.erlend.sh/assembling-community-os
blantonl 3y ago
>when all they did is provide the platform, and store the data. Is there not significant innovation and benefit that was designed and implemented in the first place that caused users to contribute their time, thought and energy? I think the real problem here is when organizations that rely on a crowd-sourced business models decide they just have to be billionaires or solve all the worlds problems with their platforms, instead of just staying true to their model. I don't see what's wrong with just running a highly successful business that makes money for it's founders and doesn't have to go out and strive every day to be the next Facebook or Google. Make no mistake. Platforms like Reddit and Stackoverflow are real, serious businesses. But why can't they exist and be a general successful business like your local mom and pop restaurant or toy store or whatever? I run RadioReference.com and Broadcastify, both which are significant businesses but also rely almost solely on crowd sourced data and content. We're wildly successful - but I've never seen the need to hire 3,000 people, or IPO, or do series raises to expand into solving world peace. Our premium subscription pricing has been the same for 15 years. I completely eliminated advertising on one of the platforms last year. We make a lot of money. We provide a lot of value to our communities, and we carefully innovate and expand to provide value. It's a nice happy life for everyone involved, and I don't have to deal with a VC who will be determined to either make a trillion dollars or torpedo my business.
deleted 3y ago
[deleted]
jodrellblank 3y ago
The problem is information hoarding. If you imagine going to a pub meeting people regularly, and the pub records everything you ever said, and then one day the pub owner says they're going to charge you for the recordings, you'd laugh. Nobody would pay. If they tried to charge you to enter, you'd go to another pub, you wouldn't lament the "loss of your culture". In fact, people don't record what they talk about in pubs because the point is the chat experience not the records of previous chats. Data isn't oil and it isn't quite sewage, it's more like quicksand or thickets of weeds growing and tangling around your feet. Like minimalists say 'stuff is bad' but stuff is useful, it's having stuff hidden in cupboards and drawers and a garage full of stuff and wanting a bigger house to hold more stuff and most of the stuff going unused because you can't bring yourself to let go of it, and companies advertising that more and newer stuff will make your life better and solve your problems, which is the biggest problem with stuff. Sufficientism might be a more appropriate name - enough stuff to make your life better and no more. Enough chat to make your life better isn't "all of it kept forever".
pc_edwin 3y ago
I believe we'll have more of these oh s** moments soon when people will finally realise why we need web3. Yes the whole space was full of scammers, charlatans but the technology and point was to create a substrate for networks on the internet. The idea that these networks and communities need to run on centralised servers is archaic. The technology exists where people should be able to own their own network (followers, subs, following, posts).
- acdha 3y ago
  web3 failed because it was a rebranding exercise by cryptocurrency holders trying to create new demand for their random numbers. Centralized servers aren’t archaic, they’re a natural outcome of how social systems work: finding communities is hard; people want to contribute their ideas, not play sysadmin; spammers and AI researchers will create enormous costs for you; etc. If you federate, you will have more time dealing with those issues than a single focused competitor and you are unlikely to see free contributions which outweigh those costs. Everything you mentioned is available now on Mastodon, and it’s really interesting to see how that works. Some people love having a small network of their friends, but a lot of people have trouble finding people they want to follow. Instances can have their own rules but dealing with abuse is now a multiparty process and since a lot of instances are run by volunteers that can be slow, unreliable, and inconsistent. Some small servers get hammered by storage and bandwidth demand but there’s no great path to monetization unless you have a ton of users willing to pay more than most people are used to paying for internet services. In general, these are social problems and there is only so much technology can do to improve them.
  - pc_edwin 3y ago
    I tend to agree but where we diverge is in the thinking that these problems cannot be solved with technology, I believe the opposite. The point of web3 is to abstract away things like sysadmin by commoditising consensus. Once a blockchain gains mass & momentum it opens up a whole new world of possibilities to hack/reinvent social media. You could use multiple different types of social media and still maintain a single identity (auth). This means you could find friends and friendlies everywhere you go. The key is the consensus layer and the ability to store & and read critical metadata. I'll give a personal example: I've been helping a friend to create a combination of Netflix and DVDs. We package the movie licenses into NFTs (MovieKeys) so when the user signs in with their wallet they can stream all the movies in they own. There are so many possibilities with this but lets focus on the social part. In theory, a social media service could scan their users wallet for MovieKeys and create a social graph based on that. Heck you could create entire forums just out of the people who owns a certain moviekey. I wont go further because we go a lot of things cooking atm. The general point is, the technology and the UX to make these things possible is just an arms length away. The entire space got ravaged by scam artists instead of trying to build real magical experiences people actually want.
2143 3y ago
> We need to revert to small, focused forums, with less anonymous, more persistent communication, run by people we trust. You're onto something. Team-BHP [1] is run exactly like this, and it seems to be working. For those wondering, it's a car-enthusiasts website based in India. They've been around for around 18 odd years I think. The moderators all have actual dayjobs. When signing up you have to write a paragraph about why you're really a petrolhead (or dieselhead because Indians love European turbo-diesels :) ), and there's a human on the other end vetting your sign-up application! Plenty, including me, have been rejected atleast once. I got in on my 2nd attempt years later. As a matter of principle they refuse to do car advertisements. I don't know how well the site is engineered but it works. Check it out. But I suspect most non-Indians (such as most people on HN) wouldn't find it that useful as it's mostly about the Indian car scene. [1] https://www.team-bhp.com/forum/ https://www.team-bhp.com/forum/
arp242 3y ago
> all they did is provide the platform, and store the data. You're seriously underestimating the effort it took to build that platform and how much effort it continues to take to keep it running well. I'm not talking about technical challenges, but social ones. It took a long time for them to get the system and incentives right (and it's still not quite right, IMHO), and it takes continued effort to keep it running well in the form of moderation and stopping abuse (and here it also doesn't quite get things right). I could bang out a "BufferUnderrun.com" in a few months; many people could. But that's not the hard part.
wefarrell 3y ago
I'm more concerned for authors of published works. Imagine writing a text book with a royalty publishing deal. Your publisher decides they're going to use your book, amongst others, to train an LLM that can answer questions on your subject, and they're not going to pay you anything. It's a legal gray area and they've got teams of lawyers whereas you do not.
saurik 3y ago
The core problem making it so difficult for this to ever actually happen is that it is 2023 and I guess you only just today somehow came to this realization as if it were new or unexpected or not something people had been saying for the past 25 years of us watching these online platforms abuse their positions of power and slowly turn the screws on people. Over the past quarter of a century of people trying to create online walled gardens of hosted content we've seen this happen over and over again, and the examples are so numerous that reddit was itself a replacement for Digg and StackOverflow for Experts Exchange. And yet, somehow, today, you suddenly woke up :(. The reality is that we live in a dystopian Eternal September where as people finally notice what is going on and leave they are just replaced by new people who don't care or simply didn't use the prior service and are attracted to the new shiny, and another 25 years from now you're going to see people making the same unapologetic "I now realize" statements. What we need to do is figure out how to actually replicate the feeling you are having in a way that doesn't require you to have spent years on a platform and then watching it die so it can be communicated to people before they bother to use a new platform, and in a way that somehow makes them willing to collectively not experience viral lock-in. (And we also need to figure out how to make people willing to accept doing that at some cost to themselves, whatever form that might take: people on HN continuously do the thing where they give up freedom for a little temporary convenience and then get angry at others for daring to suggest that something a bit harder to use or with any extra friction would ever be a sane thing for anyone to use :/.) Back in 2017 I gave a talk at Mozilla Privacy Lab called "That's How You Get Dystopia" where I just documented a ton of examples of abuse of centralized power and the reality is that every few days I just come across more stuff to add to the list... and this talk doesn't even bother with all the numerous service that simply enshittified or shuttered. https://youtu.be/vsazo-Gs7ms https://youtu.be/vsazo-Gs7ms
- akkartik 3y ago
  +100 I was saying to someone yesterday that "enshittification" is a sub-optimal coinage for something that really shouldn't need a new term, and which focuses attention on symptoms rather than root causes. If you give someone a power of attorney over your assets, you'll likely find that they start behaving less well towards you. Or if you give up agency, others will treat you like less of an agent. But what matters is not their behavior at the end but your decisions before that point.
Dalewyn 3y ago
>We need to rethink this model. Once upon a time, most people who wanted or had something to say wrote their own little website and hosted it themselves (be it in a datacenter or a server in their closet). Some even ran forums and got fancy with server-side magic because that's what nerds do. Even the kids who couldn't afford anything had free, basic hosting services to choose from (anyone remember those days?). The internet was designed as a distributed network and the denizens then were distributed. You only got as centralized as a given ISP or datacenter provider. Of course, we all know as more and more commoners came onto the internet they didn't want to bother with developing or hosting or maintaining a website or anything. They just wanted to shitpost, for free, with blackjack and hookers. And so "free" services like Reddit, Facebook, et al. came about to serve that demand. Information became centralized, because who the fuck has time to be responsible? Offload that crap! The cost of that offloading of responsibility has now come knocking with debt collectors in tow, with interest. I guess what I'm trying to say is: We don't need to rethink anything. We just need to take some god damn responsibility for ourselves. Responsibility is power, and with power you can tell commercial interests you disagree with to screw off.
ineptech 3y ago
> We need to rethink this model. This problem is inherent to client/server software, and there are really only three ways to do it: 1. The server side of client/server is centralized and run by corporations 2. The server side is decentralized, meaning everyone has their own server 3. Abandon the server, clients connect directly to each other without a server intermediating Option 3 would be ideal, but would require significant technological advances - it'll be a lo0ong time before bandwidth is cheap enough that Kim Kardashian can serve photos and movies to all of her fans direct from her phone. Option 1 is what we have now, and is terrible in a variety of ways. Option 2 would be hard but is not obviously impossible, so still our best bet - sure, it's not viable now, but it sure seems like it could be, if an iphone's worth of r&d were put in to it. I would honestly be amazed if no one at Amazon is working on such a thing, since no one would benefit more than AWS from a future in which a cloud VM becomes one of the things that most middle-class families rent monthly.
- capableweb 3y ago
  Content-addressing together with P2P and extra paid relays for those who really need it. In terms of "superstars" sharing content, if they share their image which is content-addressed and can be fetched from anyone, it's enough that one peer shares it with three others for it to be reliable enough in practice. Content like that is also usually just relevant because of recency, so large swaths of people try to access it within 24h, after that the news cycle already moved on so won't be fetched much after that.
rektide 3y ago
The world keeps going dark. What a terrible era.
jjav 3y ago
> realize a major risk with the current structure of online communication I wish this lesson could be learned once for all. A long-lived community/repository cannot be built on a proprietary platform owned by some corporation. Full stop, no exception. It can't be done. A corporation will at some point need to maximize profit extraction which will ruin it for everyone. A corporation also won't support a platform forever nor can the entity itself survive forever. A single point of failure can't last forever. > We need to decentralize our communications Look at the solutions which have lasted longest. Email & mailing lists, going strong since the 1970s. Completely decentralized, interoperability defined by open standard protocols, anyone can build interoperable clients and servers. Nobody owns it. There's no "it" to own. That is what's needed for long term viability.

by lumb63 3y ago

kiba 3y ago

Perhaps these sort of things shouldn't be for profit enterprise, given the inability of companies to not slaughter the goose that lay the golden eggs.

phailhaus 3y ago
The problem is defining "these sorts of things". StackOverflow didn't do anything evil, they created a useful website and people flocked to it voluntarily.

the_pwner224 3y ago

A decentralized system will never work because 99% of users do not care at all; the centralized systems are easier to sign up for and use. It's been demonstrated over and over and over again. Even if the underlying tech is decentralized, the community will settle around one or a few big instances (for example, Gmail and GitHub) which often end up having significant control over the trajectory of the entire ecosystem. If you run your own email server and you get put onto Google's spam list - you're fucked.

tedivm 3y ago
>If you run your own email server and you get put onto Google's spam list - you're fucked. It's even worse than that- I ran my own email server, and for some reason gmail delayed any emails from their system to outside of their system. That meant that people would send me an email but I wouldn't get it for 20 minutes. These delays don't exist when using big email providers (it stopped being a problem when I switched to Fastmail, for example) but if you're running a small server Google makes it a nightmare.
- OfSanguineFire 3y ago
  I have run my own email server off a Linode for the last decade-plus, and I have never encountered this. Most of the people with whom I correspond (I run my own business from my server) are on Gmail, and I have always received their emails instantly. If you were getting emails only 20 minutes later, I wonder if there was some server misconfiguration on your end, e.g. sending messages into graylisting delays.
- dataflow 3y ago
  I assume you didn't have anything in between Gmail and your setup? Like a forwarding layer or something?
- jethro_tell 3y ago
  Sounds like you need to allowlist google from your grey list. They have a long retry on their side. Once you tell a server to 'go away, come back later' it is really up to the server to decide when or if to retry. Additionally, If they use multiple sending IPs, you can end up grey listing again and again before they try back with a good ip. You'd either need to allowlist the big providers sending blocks or just drop grey listing all together.
  - actionfromafar 3y ago
    My gray list behaves like this. It's seldom a problem in practice, but new senders may need a "warm-up".

xen2xen1 3y ago

The risk of centralized systems was discussed long ago. The Cathedral and the Bazarr was published in 1999. None of these ideas are new. Everyone who payed any attention knew it was coming.

jacquesm 3y ago

Ever since Gracenote/CDDB it was pretty clear that this is the model. Still pissed off about that.

ethanbond 3y ago

Really? The takeaway isn’t rather that rent-seeking AI models need to figure out a way to reimburse companies and communities who’ve stored up all this capital? Seems to me SO built and delivered huge, huge amounts of value and it’s now all at risk because multibillion dollar companies are free riding.

ImPostingOnHN 3y ago
SO delivered some value, the users are the ones who delivered huge, huge amounts of value
- ethanbond 3y ago
  Right, which SO (and not some other site) managed to entice. Somehow users didn’t flock to my ethansusefulprogrammerquestionandanswers.com ¯\_(ツ)_/¯
  - ImPostingOnHN 3y ago
    It seems more likely that there, too, it was the users doing the work, not SO. the primary value on SO is generated by the users, and thus the value proposition to enticing new users is also generated by the users. SO is just a forum.
- croes 3y ago
  And companies like OpenAI will take the profit and even kill some of the users jobs.
  - ImPostingOnHN 3y ago
    and the heat death of the universe will eventually extinguish all life
    - croes 3y ago
      I think these are different time frames

dylan604 3y ago

How did newsgroups work in the days of yore? Depending on which ISP you used, you may or may not have all of the posts within a group if they even had the group. I remember paying for access to specific (can't remember the name) provider that had the most complete listing of newsgroups and had the longest retention of posts. viva a.b.m.a!!

shagie 3y ago
https://en.wikipedia.org/wiki/Network_News_Transfer_Protocol https://en.wikipedia.org/wiki/Network_News_Transfer_Protocol Originally over UUCP (Unix Unix Copy Protocol) and done via dial ups at night (when the rest of the batch transfers were done - email too with the old bang path). The two servers would exchange all the batched email and news posts that were routed to the other side. RFC 977 ( https://www.w3.org/Protocols/rfc977/rfc977 https://www.w3.org/Protocols/rfc977/rfc977 ) has an example of how files are copied between the two systems (section 4.6) including fetching and receiving mail. Note that not all posts outbound are necessarily of interest to the other server. An IHAVE message could come back with either a "I want it" type response or a "not interested" > The IHAVE command informs the server that the client has an article whose id is <messageid>. If the server desires a copy of that article, it will return a response instructing the client to send the entire article. If the server does not want the article (if, for example, the server already has a copy of it), a response indicating that the article is not wanted will be returned. That's how some of the moderation worked - your server would say "I don't want anything that came by way of X host" or "not interested in that newsgroup." One of the amusing things to me (looking back at this), if you're familiar with HTTP response codes, you'll likely get most of the way through the NNTP ones. 200 server ready - posting allowed 400 service discontinued 411 no such news group 500 command not recognized I'd also suggest a read of RFC 850 ( https://www.w3.org/Protocols/rfc850/rfc850.html https://www.w3.org/Protocols/rfc850/rfc850.html ) for some other background and section 5: The News Propagation Algorithm
- dylan604 3y ago
  But how did all of this line up within the federated or not conversation? If each ISP could host their own version, that doesn't sound federated. But who was in control of the "main" source of truth type of version?

b3morales 3y ago

This was one of the promises originally of Stack Overflow: all the content is Creative Commons licensed so that if they "turned evil" (I believe it was Joel that put it this way) the community could, in a way, create a fork. https://web.archive.org/web/20230203170609/https://stackoverflow.blog/2009/06/04/stack-overflow-creative-commons-data-dump/ https://web.archive.org/web/20230203170609/https://stackover... Unfortunately the dumps themselves are not a legal requirement, just a gentleman's agreement, so realistically exercising this ability was still at the whim of the company.

wahnfrieden 3y ago
So it's time to community fork?
- b3morales 3y ago
  Assuming that the linked post is accurate and that the "approval from senior leadership" to turn the dump back on does not come...then yes, I would say so. Actually there is already Codidact, although if I recall correctly they explicitly ruled out importing SE data when they started up. https://codidact.org https://codidact.org
  - deleted 3y ago
    [deleted]
  - Wowfunhappy 3y ago
    Yeah, Codidact isn't a "fork" because they don't use the SE data.
    - shagie 3y ago
      It’s a fork of the community rather than the data and content.
      - sitkack 3y ago

scarface_74 3y ago

> when all they did is provide the platform, and store the data And all Google did was build a search engine.

nightski 3y ago

I like the idea of decentralized but I'd suggest you don't need to go fully decentralized where every peer has a full copy. Actually I kind of like the Bitcoin approach in that you have the ability to create a full peer, but most people do not. This would allow some decentralization and reduce risk, but not burden everybody with running a full peer.

jupp0r 3y ago

Incentives in this structure go both ways though, ideally keeping everything in symbiotic balance. Companies that alienate their users tend to not do well shortly after.

ilyt 3y ago

That doesn't stop grumpy admin of a federated instance to just do exactly same thing. > run by people we trust People change, or retire, just like corporation goals change. Focusing on more independent is not enough. If you want truly unbreakable stuff first part of the puzzle is saving user's handle and identity in a way that can't be removed. Then finding out a way to link that to their content so when place of hosting it goes away people can follow to the new place Then just have all of that content be signed by that identity so users can verify that it is really that person. And I can't believe I'm saying that unironically but blockchain might just be the solution for that. Something like immutable log of: * user declaring "I'm jeff@example.com, here are my public keys". Servers then validate via DNS record or some .well-known location entry whether user is allowed to declare they are from @example.com * user declaring "behold! jeff@example.com stuff is <here>, and <here> and <here> are addresses for various federation systems". Only passes if that request is signed with above privkey of course * user declaring "behold! My new public key is X and Y. And Z key is revoked!" * user declaring "behold! I am now george.effluent@company.com! Re-does checks but for new domain and users previously subscribed to jeff@example com get served redirect". etc. Then when server admin inevitably goes rogue you can take your posts and subscribers and go somewhere else. And when @example.com owner decides "well I'm just gonna to redirect stuff to ads", you can just change your handle and direct people to right place, and other handle is forever taken.

deleted 3y ago

[deleted]

deleted 3y ago

[deleted]

einpoklum 3y ago

> The thought and knowledge of communities and users need to belong to those communities and users. I would say: Need to be a public resource, belonging to no-one, i.e. no person, or group, or company should have legitimacy in denying access to it. They should all be considered _trustees_ of such a resource. > when all they did is provide the platform, and store the data. To be fair, SE Inc. did a lot more than provide the platform. A lot of development and design work, publication, a bit of the curation work, etc. I don't like how they behave but let's give them what they're due. --- Also note the ongoing Moderator Strike (!): https://meta.stackexchange.com/q/389811/196834 https://meta.stackexchange.com/q/389811/196834

briffle 3y ago

I'd argue its not just forums, but other key parts of the internet. Like Microsoft training co-pilot AI on github code, but not following the licensing of some code they straight up copy and suggest. I'm kind of curious what is next.

erlend_sh 3y ago

Agreed, and working on it: https://blog.erlend.sh/assembling-community-os https://blog.erlend.sh/assembling-community-os

blantonl 3y ago

>when all they did is provide the platform, and store the data. Is there not significant innovation and benefit that was designed and implemented in the first place that caused users to contribute their time, thought and energy? I think the real problem here is when organizations that rely on a crowd-sourced business models decide they just have to be billionaires or solve all the worlds problems with their platforms, instead of just staying true to their model. I don't see what's wrong with just running a highly successful business that makes money for it's founders and doesn't have to go out and strive every day to be the next Facebook or Google. Make no mistake. Platforms like Reddit and Stackoverflow are real, serious businesses. But why can't they exist and be a general successful business like your local mom and pop restaurant or toy store or whatever? I run RadioReference.com and Broadcastify, both which are significant businesses but also rely almost solely on crowd sourced data and content. We're wildly successful - but I've never seen the need to hire 3,000 people, or IPO, or do series raises to expand into solving world peace. Our premium subscription pricing has been the same for 15 years. I completely eliminated advertising on one of the platforms last year. We make a lot of money. We provide a lot of value to our communities, and we carefully innovate and expand to provide value. It's a nice happy life for everyone involved, and I don't have to deal with a VC who will be determined to either make a trillion dollars or torpedo my business.

deleted 3y ago

[deleted]

jodrellblank 3y ago

The problem is information hoarding. If you imagine going to a pub meeting people regularly, and the pub records everything you ever said, and then one day the pub owner says they're going to charge you for the recordings, you'd laugh. Nobody would pay. If they tried to charge you to enter, you'd go to another pub, you wouldn't lament the "loss of your culture". In fact, people don't record what they talk about in pubs because the point is the chat experience not the records of previous chats. Data isn't oil and it isn't quite sewage, it's more like quicksand or thickets of weeds growing and tangling around your feet. Like minimalists say 'stuff is bad' but stuff is useful, it's having stuff hidden in cupboards and drawers and a garage full of stuff and wanting a bigger house to hold more stuff and most of the stuff going unused because you can't bring yourself to let go of it, and companies advertising that more and newer stuff will make your life better and solve your problems, which is the biggest problem with stuff. Sufficientism might be a more appropriate name - enough stuff to make your life better and no more. Enough chat to make your life better isn't "all of it kept forever".

pc_edwin 3y ago

I believe we'll have more of these oh s** moments soon when people will finally realise why we need web3. Yes the whole space was full of scammers, charlatans but the technology and point was to create a substrate for networks on the internet. The idea that these networks and communities need to run on centralised servers is archaic. The technology exists where people should be able to own their own network (followers, subs, following, posts).

acdha 3y ago
web3 failed because it was a rebranding exercise by cryptocurrency holders trying to create new demand for their random numbers. Centralized servers aren’t archaic, they’re a natural outcome of how social systems work: finding communities is hard; people want to contribute their ideas, not play sysadmin; spammers and AI researchers will create enormous costs for you; etc. If you federate, you will have more time dealing with those issues than a single focused competitor and you are unlikely to see free contributions which outweigh those costs. Everything you mentioned is available now on Mastodon, and it’s really interesting to see how that works. Some people love having a small network of their friends, but a lot of people have trouble finding people they want to follow. Instances can have their own rules but dealing with abuse is now a multiparty process and since a lot of instances are run by volunteers that can be slow, unreliable, and inconsistent. Some small servers get hammered by storage and bandwidth demand but there’s no great path to monetization unless you have a ton of users willing to pay more than most people are used to paying for internet services. In general, these are social problems and there is only so much technology can do to improve them.
- pc_edwin 3y ago
  I tend to agree but where we diverge is in the thinking that these problems cannot be solved with technology, I believe the opposite. The point of web3 is to abstract away things like sysadmin by commoditising consensus. Once a blockchain gains mass & momentum it opens up a whole new world of possibilities to hack/reinvent social media. You could use multiple different types of social media and still maintain a single identity (auth). This means you could find friends and friendlies everywhere you go. The key is the consensus layer and the ability to store & and read critical metadata. I'll give a personal example: I've been helping a friend to create a combination of Netflix and DVDs. We package the movie licenses into NFTs (MovieKeys) so when the user signs in with their wallet they can stream all the movies in they own. There are so many possibilities with this but lets focus on the social part. In theory, a social media service could scan their users wallet for MovieKeys and create a social graph based on that. Heck you could create entire forums just out of the people who owns a certain moviekey. I wont go further because we go a lot of things cooking atm. The general point is, the technology and the UX to make these things possible is just an arms length away. The entire space got ravaged by scam artists instead of trying to build real magical experiences people actually want.

2143 3y ago

> We need to revert to small, focused forums, with less anonymous, more persistent communication, run by people we trust. You're onto something. Team-BHP [1] is run exactly like this, and it seems to be working. For those wondering, it's a car-enthusiasts website based in India. They've been around for around 18 odd years I think. The moderators all have actual dayjobs. When signing up you have to write a paragraph about why you're really a petrolhead (or dieselhead because Indians love European turbo-diesels :) ), and there's a human on the other end vetting your sign-up application! Plenty, including me, have been rejected atleast once. I got in on my 2nd attempt years later. As a matter of principle they refuse to do car advertisements. I don't know how well the site is engineered but it works. Check it out. But I suspect most non-Indians (such as most people on HN) wouldn't find it that useful as it's mostly about the Indian car scene. [1] https://www.team-bhp.com/forum/ https://www.team-bhp.com/forum/

arp242 3y ago

> all they did is provide the platform, and store the data. You're seriously underestimating the effort it took to build that platform and how much effort it continues to take to keep it running well. I'm not talking about technical challenges, but social ones. It took a long time for them to get the system and incentives right (and it's still not quite right, IMHO), and it takes continued effort to keep it running well in the form of moderation and stopping abuse (and here it also doesn't quite get things right). I could bang out a "BufferUnderrun.com" in a few months; many people could. But that's not the hard part.

wefarrell 3y ago

I'm more concerned for authors of published works. Imagine writing a text book with a royalty publishing deal. Your publisher decides they're going to use your book, amongst others, to train an LLM that can answer questions on your subject, and they're not going to pay you anything. It's a legal gray area and they've got teams of lawyers whereas you do not.

saurik 3y ago

The core problem making it so difficult for this to ever actually happen is that it is 2023 and I guess you only just today somehow came to this realization as if it were new or unexpected or not something people had been saying for the past 25 years of us watching these online platforms abuse their positions of power and slowly turn the screws on people. Over the past quarter of a century of people trying to create online walled gardens of hosted content we've seen this happen over and over again, and the examples are so numerous that reddit was itself a replacement for Digg and StackOverflow for Experts Exchange. And yet, somehow, today, you suddenly woke up :(. The reality is that we live in a dystopian Eternal September where as people finally notice what is going on and leave they are just replaced by new people who don't care or simply didn't use the prior service and are attracted to the new shiny, and another 25 years from now you're going to see people making the same unapologetic "I now realize" statements. What we need to do is figure out how to actually replicate the feeling you are having in a way that doesn't require you to have spent years on a platform and then watching it die so it can be communicated to people before they bother to use a new platform, and in a way that somehow makes them willing to collectively not experience viral lock-in. (And we also need to figure out how to make people willing to accept doing that at some cost to themselves, whatever form that might take: people on HN continuously do the thing where they give up freedom for a little temporary convenience and then get angry at others for daring to suggest that something a bit harder to use or with any extra friction would ever be a sane thing for anyone to use :/.) Back in 2017 I gave a talk at Mozilla Privacy Lab called "That's How You Get Dystopia" where I just documented a ton of examples of abuse of centralized power and the reality is that every few days I just come across more stuff to add to the list... and this talk doesn't even bother with all the numerous service that simply enshittified or shuttered. https://youtu.be/vsazo-Gs7ms https://youtu.be/vsazo-Gs7ms

akkartik 3y ago
+100 I was saying to someone yesterday that "enshittification" is a sub-optimal coinage for something that really shouldn't need a new term, and which focuses attention on symptoms rather than root causes. If you give someone a power of attorney over your assets, you'll likely find that they start behaving less well towards you. Or if you give up agency, others will treat you like less of an agent. But what matters is not their behavior at the end but your decisions before that point.

Dalewyn 3y ago

>We need to rethink this model. Once upon a time, most people who wanted or had something to say wrote their own little website and hosted it themselves (be it in a datacenter or a server in their closet). Some even ran forums and got fancy with server-side magic because that's what nerds do. Even the kids who couldn't afford anything had free, basic hosting services to choose from (anyone remember those days?). The internet was designed as a distributed network and the denizens then were distributed. You only got as centralized as a given ISP or datacenter provider. Of course, we all know as more and more commoners came onto the internet they didn't want to bother with developing or hosting or maintaining a website or anything. They just wanted to shitpost, for free, with blackjack and hookers. And so "free" services like Reddit, Facebook, et al. came about to serve that demand. Information became centralized, because who the fuck has time to be responsible? Offload that crap! The cost of that offloading of responsibility has now come knocking with debt collectors in tow, with interest. I guess what I'm trying to say is: We don't need to rethink anything. We just need to take some god damn responsibility for ourselves. Responsibility is power, and with power you can tell commercial interests you disagree with to screw off.

ineptech 3y ago

> We need to rethink this model. This problem is inherent to client/server software, and there are really only three ways to do it: 1. The server side of client/server is centralized and run by corporations 2. The server side is decentralized, meaning everyone has their own server 3. Abandon the server, clients connect directly to each other without a server intermediating Option 3 would be ideal, but would require significant technological advances - it'll be a lo0ong time before bandwidth is cheap enough that Kim Kardashian can serve photos and movies to all of her fans direct from her phone. Option 1 is what we have now, and is terrible in a variety of ways. Option 2 would be hard but is not obviously impossible, so still our best bet - sure, it's not viable now, but it sure seems like it could be, if an iphone's worth of r&d were put in to it. I would honestly be amazed if no one at Amazon is working on such a thing, since no one would benefit more than AWS from a future in which a cloud VM becomes one of the things that most middle-class families rent monthly.

capableweb 3y ago
Content-addressing together with P2P and extra paid relays for those who really need it. In terms of "superstars" sharing content, if they share their image which is content-addressed and can be fetched from anyone, it's enough that one peer shares it with three others for it to be reliable enough in practice. Content like that is also usually just relevant because of recency, so large swaths of people try to access it within 24h, after that the news cycle already moved on so won't be fetched much after that.

rektide 3y ago

The world keeps going dark. What a terrible era.

jjav 3y ago

> realize a major risk with the current structure of online communication I wish this lesson could be learned once for all. A long-lived community/repository cannot be built on a proprietary platform owned by some corporation. Full stop, no exception. It can't be done. A corporation will at some point need to maximize profit extraction which will ruin it for everyone. A corporation also won't support a platform forever nor can the entity itself survive forever. A single point of failure can't last forever. > We need to decentralize our communications Look at the solutions which have lasted longest. Email & mailing lists, going strong since the 1970s. Completely decentralized, interoperability defined by open standard protocols, anyone can build interoperable clients and servers. Nobody owns it. There's no "it" to own. That is what's needed for long term viability.