8 ms·

Federated systems just don't work in practice because of inherent problems with them that don't have good technical&social solutions yet. Everything on the Inte

by dpc_pw 6y ago

Federated systems just don't work in practice because of inherent problems with them that don't have good technical&social solutions yet. Everything on the Internet (including Internet itself) degenerated from federation into monopolistic centralized services for a reason, and somehow people keep ignoring the evidence, and just keep trying harder. These problems are so fundamental and universal that you pretty much need "a new form of Internet" that solves them, so then apps running in it can compete with centralized services. So far the best stab at the problem is Urbit, but mostly because almost nobody else is even trying. Handshake and Holochains are another interesting projects in this space, but more of a pieces of potentially bigger picture.

IanDrake 6y ago
You would more likely need a new form of people. Coke is number one for a reason. Sure there is Pepsi, and yes there are small soda makers that either get acquired or close, but not too many people make their own soda. For the same reason, there won't be a federated social network... Unless, people don't even realize they're using one, but that gets back to your point, we'd need a new form of internet.
zrail 6y ago
The current shape of the internet itself (BGP peering among autonomous systems, a decent number of large tier 1 ISPs, etc) is, on its face, a relatively successful federation. The applications running atop that federation are, as you say, centralized, but much of that is because residential ISPs forbid customers forbid servers and even if they didn’t (they haven’t always) residential customers are prone to accidentally running botnets of various types.
Mc91 6y ago
> Everything on the Internet (including Internet itself) degenerated from federation into monopolistic centralized services for a reason, and somehow people keep ignoring the evidence, and just keep trying harder. Sure there's a reason federated networks like Usenet died - they were killed off by a combination of the old/new AT&T/Verizon duopoly and various government organizations ( https://www.cnet.com/news/n-y-attorney-general-forces-isps-to-curb-usenet-access/ https://www.cnet.com/news/n-y-attorney-general-forces-isps-t... ). Yes, it happened for a reason - the old Bell last-mile monopoly and active government measures to quash it.
- rootusrootus 6y ago
  Usenet died from a combination of web forums being a better experience for most people, and a saturation with warez binary traffic becoming so expensive that ISPs did not want to pay for it.
throwanem 6y ago
If these problems are so obviously ineluctable, why talk around them instead of detailing or even naming any?
smichel17 6y ago
I am not sure this is self-evident; it could be that there is no end state, but rather a cycle of centralized and federated systems, and we simply haven't seen enough of the cycle to identify it, yet. As a parallel, consider systems of government in the last 2500 years or so.
- walterbell 6y ago
  Could you share examples from the governance cycle?
  - nine_k 6y ago
    A cycle of democracies / republics / authoritarian regimes? A cycle of smaller independent states / large empires? This has repeated a lot in the history of the western civilization.
  - smichel17 6y ago
    I was thinking of the rise and fall of democracy from before the Roman empire and through the present day. Pick any time and look back only a few centuries, and the answer to "What is the natural end state of government?" looks very different. Zoom out a bit and it's clear that if there is an end state, we haven't reached it yet. Please note: I am being very careful with the claims I am making here. I have a little more than a layperson's understanding of Rome's history (years of Latin classes paying off at last!) but not nearly enough to call myself an authority on governments through the ages. My point -- my only[0] point -- is that "The Internet started off federated and has become centralized / nobody has gotten federation to stick yet" is not enough of an argument to support "federation is doomed to failure". My comment does not totally disagree with its parent, which seems to think that alternative approaches may succeed. We certainly agree that it's important to learn from the past. I'm having trouble wrapping this up, but don't want to spend more time, so I'll leave it on that somewhat unfinished note. [0] edit: I suppose there is also a secondary point that systems design is, like governance, an inherently social issue. This makes any analysis a lot more complex than if we restrict ourselves to purely technical merits.
F3nd0 6y ago
> So far the best stab at the problem is Urbit, but mostly because almost nobody else is even trying. What about GNUnet? They’re trying to build pretty much what you described, from the ground up. https://gnunet.org/en/ https://gnunet.org/en/
- dpc_pw 6y ago
  I'm aware of GNUnet, and been following it loosely for too long to remember. It's been around for like 2 decades? Superficially it looks to me like bunch of theoreticians overcomplicated everything, and it will never be practical. Probably great project to advance your academic carrier though. But maybe I'm wrong.
reificator 6y ago
> Federated systems just don't work in practice because of inherent problems with them that don't have good technical&social solutions yet. Everything on the Internet (including Internet itself) degenerated from federation into monopolistic centralized services for a reason, and somehow people keep ignoring the evidence, and just keep trying harder. Email would beg to differ. Google and Microsoft may have a lot of users, but interoperability is astoundingly high. Yes, there is a high barrier to sending emails from a new provider, and yes, Google/Microsoft have a large chunk of the market. Still, there's little barrier to using ProtonMail or Fastmail or whatever free host you want, and everything interoperates shockingly well. You don't typically have to think about "what if this person has gmail?" or "I don't know if I can send to that host from outlook".
- clusterfish 6y ago
  Email was already too big to fail by the time Facebook and Google got big enough to be able to kill it. Looking at stuff created in the last 10-15 years, are there any federation successes? I can't think of any. (Blockchain excluded for the reason stated in the article)
  - lokedhs 6y ago
    What's the definition of success? I spend far too much time on the Fediverse (mastodon) and if success is measured by the amount of time I spend there, it's the most successful service I use. Of course, there are few if any political leaders or CEO's posting there, but that's not a definition of success to me. I have thought about what would happen if someone like, say, Musk would proclaim that they're quitting Twitter and setting up a Mastodon instance instead. I'm not sure the Fediverse would be better off if that happened.
    - clusterfish 6y ago
      Wide adoption. Especially for social network type apps, which are rather useless without my friends in them.
kstrauser 6y ago
You mean, other than Mastodon, which has a few million users and growing.
- bsagdiyev 6y ago
  My grandma will never use Mastodon, but she uses Facebook. Tech isn’t the only audience you need.
  - kixiQu 6y ago
    I think tech is a tiny minority on Mastodon. From what I can see there are far more queer folk and lefty ppl. Your grandma will never use Mastodon because there are no network effects luring her there, but the UX is about as good as a normal centralized social site.
  - riffic 6y ago
    sorry, but your grandmother doesn't have a few hundred thousand to a few million followers. media organizations, pols, and public sector institutions do. Your grandma doesn't need to use Mastodon and it's not about her. It's about those who will be followed and have something important to transmit.
- ApolloFortyNine 6y ago
  Mastodon is proof at least that federation doesn't fix the issue of censorship. There's a common ban list in Mastodon that if you don't include, you will not be able to join the "main" Mastodon cluster. And if your not part of the main group, what's even the point of federation? It's practically "servers" at that point.
  - throwaway33339 6y ago
    It does fix the issue of censorship, in that 'blacklisted' instances can still have a world of their own and use the technology or protocol. No amount of technology is going to fix the 'a majority of people don't like you or interact with you' problem though.
  - 6y ago
shmerl 6y ago
E-mail doesn't work too? Federation works, if some walled garden obsessive lock-in freaks (like Compuserve and AOL of old or today's analogies of Facebook and the like) don't manage to mess things up.
- TylerE 6y ago
  Email sucks. Spam is ramptant. Running a server is a nightmare. Standards are a tar pit. It "works" only for fairly loose definitions of "works".
  - sliken 6y ago
    Have you tried to setup a mail server these days? There's plenty of nearly turn key solutions. Even from scratch on Debian, Fedora, or Ubuntu is pretty easy these days, a few apt-gets and a few additional tweaks and away you go. Using spam-assassin, clamav, and a few recommended tweaks (greylisting, HELO rules, blocking .ads, .party, and friends) and I'm down near zero spam.
    - danielheath 6y ago
      How's your delivery? Anything that runs the risk of my mail getting blackholed by a clients provider is a nonstarter (even if it's only a few days every few years).
      - sliken 6y ago
        Been fine, I have noticed others with terrible results. My best guess is that large email provides use domain age. So new domains have terrible problems with delivery. My domain is many years old.
  - shmerl 6y ago
    Still, it works.
  - pfraze 6y ago
    Centralized servers are just as hard to run, if not harder. You just don't experience those ops yourself without working there.
samatman 6y ago
Urbit is also not a federated system, but rather, a true distributed system. Each urbit instance is a peer. The stars and galaxies provide discovery and initial routing, which is a pretty neat solution, they're basically fancy DNS servers. What I think is exciting about urbit, is that each identity can host services. So you can launch a chat room, and it's yours, other peers can show up and say things in it, but this is very different from federation, where there is a sharp distinction between administrators and users. Anyone who comments in your chat room can have a chat of their own, and the cryptographic identity of you and them is identical and consistent, wherever in the fleet you may encounter each other. Stability and performance have improved, but there's a long way to go before a smooth one-click user experience is a reality. But the basic design is solid, and it has a chance precisely because it isn't federated, but distributed. Secure Scuttlebutt is another truly distributed system, of very different design; it's locked in some bad architecture but I found it quite pleasant to interact with. Don't expect anything useful from Holochain, ever. Just... I don't want to get into inside baseball, but, nah. Never gonna happen.
- dpc_pw 6y ago
  Yeah, for me a federated system taken to its logical conclusion turns into p2p system. SSB is neat, but ineed could use rearchitecting. I could imagine ssb-like system + handshake to work quite OK for certain things. Thanks for info on Holochain. I've been only lurking in there and reading docs, so my understanding is rather shallow and theoretical.
trabant00 6y ago
> Federated systems just don't work in practice because of inherent problems with them The entire form of current global human society is federated so you can't simply state it does not work. For the internet so far we have a few empires that battle it out, but I don't think they are manageable long term and this has started to become apparent.
deleted 6y ago
[deleted]
mariusor 6y ago
> because of inherent problems with them that don't have good technical&social solutions yet Would you care to mention them?

by dpc_pw 6y ago

IanDrake 6y ago

You would more likely need a new form of people. Coke is number one for a reason. Sure there is Pepsi, and yes there are small soda makers that either get acquired or close, but not too many people make their own soda. For the same reason, there won't be a federated social network... Unless, people don't even realize they're using one, but that gets back to your point, we'd need a new form of internet.

zrail 6y ago

The current shape of the internet itself (BGP peering among autonomous systems, a decent number of large tier 1 ISPs, etc) is, on its face, a relatively successful federation. The applications running atop that federation are, as you say, centralized, but much of that is because residential ISPs forbid customers forbid servers and even if they didn’t (they haven’t always) residential customers are prone to accidentally running botnets of various types.

Mc91 6y ago

> Everything on the Internet (including Internet itself) degenerated from federation into monopolistic centralized services for a reason, and somehow people keep ignoring the evidence, and just keep trying harder. Sure there's a reason federated networks like Usenet died - they were killed off by a combination of the old/new AT&T/Verizon duopoly and various government organizations ( https://www.cnet.com/news/n-y-attorney-general-forces-isps-to-curb-usenet-access/ https://www.cnet.com/news/n-y-attorney-general-forces-isps-t... ). Yes, it happened for a reason - the old Bell last-mile monopoly and active government measures to quash it.

rootusrootus 6y ago
Usenet died from a combination of web forums being a better experience for most people, and a saturation with warez binary traffic becoming so expensive that ISPs did not want to pay for it.

throwanem 6y ago

If these problems are so obviously ineluctable, why talk around them instead of detailing or even naming any?

smichel17 6y ago

I am not sure this is self-evident; it could be that there is no end state, but rather a cycle of centralized and federated systems, and we simply haven't seen enough of the cycle to identify it, yet. As a parallel, consider systems of government in the last 2500 years or so.

walterbell 6y ago
Could you share examples from the governance cycle?
- nine_k 6y ago
  A cycle of democracies / republics / authoritarian regimes? A cycle of smaller independent states / large empires? This has repeated a lot in the history of the western civilization.
- smichel17 6y ago
  I was thinking of the rise and fall of democracy from before the Roman empire and through the present day. Pick any time and look back only a few centuries, and the answer to "What is the natural end state of government?" looks very different. Zoom out a bit and it's clear that if there is an end state, we haven't reached it yet. Please note: I am being very careful with the claims I am making here. I have a little more than a layperson's understanding of Rome's history (years of Latin classes paying off at last!) but not nearly enough to call myself an authority on governments through the ages. My point -- my only[0] point -- is that "The Internet started off federated and has become centralized / nobody has gotten federation to stick yet" is not enough of an argument to support "federation is doomed to failure". My comment does not totally disagree with its parent, which seems to think that alternative approaches may succeed. We certainly agree that it's important to learn from the past. I'm having trouble wrapping this up, but don't want to spend more time, so I'll leave it on that somewhat unfinished note. [0] edit: I suppose there is also a secondary point that systems design is, like governance, an inherently social issue. This makes any analysis a lot more complex than if we restrict ourselves to purely technical merits.

F3nd0 6y ago

> So far the best stab at the problem is Urbit, but mostly because almost nobody else is even trying. What about GNUnet? They’re trying to build pretty much what you described, from the ground up. https://gnunet.org/en/ https://gnunet.org/en/

dpc_pw 6y ago
I'm aware of GNUnet, and been following it loosely for too long to remember. It's been around for like 2 decades? Superficially it looks to me like bunch of theoreticians overcomplicated everything, and it will never be practical. Probably great project to advance your academic carrier though. But maybe I'm wrong.

reificator 6y ago

> Federated systems just don't work in practice because of inherent problems with them that don't have good technical&social solutions yet. Everything on the Internet (including Internet itself) degenerated from federation into monopolistic centralized services for a reason, and somehow people keep ignoring the evidence, and just keep trying harder. Email would beg to differ. Google and Microsoft may have a lot of users, but interoperability is astoundingly high. Yes, there is a high barrier to sending emails from a new provider, and yes, Google/Microsoft have a large chunk of the market. Still, there's little barrier to using ProtonMail or Fastmail or whatever free host you want, and everything interoperates shockingly well. You don't typically have to think about "what if this person has gmail?" or "I don't know if I can send to that host from outlook".

clusterfish 6y ago
Email was already too big to fail by the time Facebook and Google got big enough to be able to kill it. Looking at stuff created in the last 10-15 years, are there any federation successes? I can't think of any. (Blockchain excluded for the reason stated in the article)
- lokedhs 6y ago
  What's the definition of success? I spend far too much time on the Fediverse (mastodon) and if success is measured by the amount of time I spend there, it's the most successful service I use. Of course, there are few if any political leaders or CEO's posting there, but that's not a definition of success to me. I have thought about what would happen if someone like, say, Musk would proclaim that they're quitting Twitter and setting up a Mastodon instance instead. I'm not sure the Fediverse would be better off if that happened.
  - clusterfish 6y ago
    Wide adoption. Especially for social network type apps, which are rather useless without my friends in them.

kstrauser 6y ago

You mean, other than Mastodon, which has a few million users and growing.

bsagdiyev 6y ago
My grandma will never use Mastodon, but she uses Facebook. Tech isn’t the only audience you need.
- kixiQu 6y ago
  I think tech is a tiny minority on Mastodon. From what I can see there are far more queer folk and lefty ppl. Your grandma will never use Mastodon because there are no network effects luring her there, but the UX is about as good as a normal centralized social site.
- riffic 6y ago
  sorry, but your grandmother doesn't have a few hundred thousand to a few million followers. media organizations, pols, and public sector institutions do. Your grandma doesn't need to use Mastodon and it's not about her. It's about those who will be followed and have something important to transmit.
ApolloFortyNine 6y ago
Mastodon is proof at least that federation doesn't fix the issue of censorship. There's a common ban list in Mastodon that if you don't include, you will not be able to join the "main" Mastodon cluster. And if your not part of the main group, what's even the point of federation? It's practically "servers" at that point.
- throwaway33339 6y ago
  It does fix the issue of censorship, in that 'blacklisted' instances can still have a world of their own and use the technology or protocol. No amount of technology is going to fix the 'a majority of people don't like you or interact with you' problem though.
- 6y ago

shmerl 6y ago

E-mail doesn't work too? Federation works, if some walled garden obsessive lock-in freaks (like Compuserve and AOL of old or today's analogies of Facebook and the like) don't manage to mess things up.

TylerE 6y ago
Email sucks. Spam is ramptant. Running a server is a nightmare. Standards are a tar pit. It "works" only for fairly loose definitions of "works".
- sliken 6y ago
  Have you tried to setup a mail server these days? There's plenty of nearly turn key solutions. Even from scratch on Debian, Fedora, or Ubuntu is pretty easy these days, a few apt-gets and a few additional tweaks and away you go. Using spam-assassin, clamav, and a few recommended tweaks (greylisting, HELO rules, blocking .ads, .party, and friends) and I'm down near zero spam.
  - danielheath 6y ago
    How's your delivery? Anything that runs the risk of my mail getting blackholed by a clients provider is a nonstarter (even if it's only a few days every few years).
    - sliken 6y ago
      Been fine, I have noticed others with terrible results. My best guess is that large email provides use domain age. So new domains have terrible problems with delivery. My domain is many years old.
- shmerl 6y ago
  Still, it works.
- pfraze 6y ago
  Centralized servers are just as hard to run, if not harder. You just don't experience those ops yourself without working there.

samatman 6y ago

Urbit is also not a federated system, but rather, a true distributed system. Each urbit instance is a peer. The stars and galaxies provide discovery and initial routing, which is a pretty neat solution, they're basically fancy DNS servers. What I think is exciting about urbit, is that each identity can host services. So you can launch a chat room, and it's yours, other peers can show up and say things in it, but this is very different from federation, where there is a sharp distinction between administrators and users. Anyone who comments in your chat room can have a chat of their own, and the cryptographic identity of you and them is identical and consistent, wherever in the fleet you may encounter each other. Stability and performance have improved, but there's a long way to go before a smooth one-click user experience is a reality. But the basic design is solid, and it has a chance precisely because it isn't federated, but distributed. Secure Scuttlebutt is another truly distributed system, of very different design; it's locked in some bad architecture but I found it quite pleasant to interact with. Don't expect anything useful from Holochain, ever. Just... I don't want to get into inside baseball, but, nah. Never gonna happen.

dpc_pw 6y ago
Yeah, for me a federated system taken to its logical conclusion turns into p2p system. SSB is neat, but ineed could use rearchitecting. I could imagine ssb-like system + handshake to work quite OK for certain things. Thanks for info on Holochain. I've been only lurking in there and reading docs, so my understanding is rather shallow and theoretical.

trabant00 6y ago

> Federated systems just don't work in practice because of inherent problems with them The entire form of current global human society is federated so you can't simply state it does not work. For the internet so far we have a few empires that battle it out, but I don't think they are manageable long term and this has started to become apparent.

deleted 6y ago

[deleted]

mariusor 6y ago

> because of inherent problems with them that don't have good technical&social solutions yet Would you care to mention them?