16 ms·

The unrealized potential of federation

194 points by als0 6y ago

zaksingh 6y ago
The biggest challenge I see with federated systems is the discovery process for finding a node you want to be associated with. This is a hard choice - will this server still be running in 6 months? How is it funded? Is it being actively maintained? These questions are too much for laypeople, and I can't imagine a scenario in which your friendly local system admin is your service provider for a critical service in your life like Facebook. Could you imagine email working that way? I see federated systems naturally ending in centralization in the form of a couple big megacorps that the average person trusts to not shut down tomorrow. Convenience and ease of use will win out. It happened with email, it arguably happened with xmpp back in the day. Change my view?
- ddevault 6y ago
  There's no evidence of this happening with, say, Mastodon. The culture is different, and isn't changing. If we know what we want and what we don't want from the start, we can keep centralization from gaining a foothold.
- kixiQu 6y ago
  You can migrate a Mastodon account and the old server will forward on your follows.
warkdarrior 6y ago
> "This [federation] system also makes it hard for marketing and spam to get a foothold" > "You’re certainly familiar with another federation which is not based on ActivityPub: email." Self-contradicting article?
- ddevault 6y ago
  Email is a bit different from the new federations we're seeing today: there's less weaker guarantees of identity, and there's less emphasis on choosing what other instances you federate with. In the example of Mastodon, the cost of setting up a new server is also too high to make spam effective, since you need a persistent, reachable account in order to interact with other instances on behalf of it. And the social factor counts, too, newer federations like this tend to be opposed to corporate influence in general and this makes it hard for it to get the initial foothold. Email has a lot of spam, but a federated system which keeps this in mind from the outset needn't suffer the same fate. EDIT: Incorporating the replies to my comment, I also want to mention that the lack of large, centralized players like gmail also prevents spam. If there are a lot of intances, and they're all small, then cutting one off for bad behavior is a lot less difficult. Instance admins can be held accountable for the behavior of their users.
  - Jasper_ 6y ago
    That would imply that centralized solutions would have even less spam, since they have much stronger guarantees of identity. And yet sites like Twitter has plenty of spam. Even Hacker News gets spam! And this is not a popular place in the grand scheme of things. How is distributed identity solved in the long term? Nobody would run a server to communicate spam; that's rarely how email spam is sent these days. Most of the spam I get is from accounts hosted on GMail and Yahoo!. So the weakest link is a single server that's community-respected, but doesn't have adequate spam protection. Federation makes that much harder, because the whole point is that every server is run separately. Mastodon gets no spam today because it's smaller than Hacker News. There's relatively nobody on there.
    - kzrdude 6y ago
      I'm worried how mastodon is going to scale (in terms of moderation) - maybe some instance can experient with crowd sourced moderation.
      - indigochill 6y ago
        Part of the point, as I understand it, is that moderation scales more-or-less linearly because each instance is ideally human-scale (that is, maybe having hundreds of members as opposed to millions) and responsible for its own moderation, unlike sites like Twitter or Facebook which have to pay a relatively small team of moderators to moderate millions of people's communications. There's nothing saying any particular instance will moderate, but if they don't and it becomes a problem they'll quickly find themselves cut off from federating with other instances.
    - riffic 6y ago
      > How is distributed identity solved in the long term? Same way email handles it -- through the use of the domain name system. You'd expect that an a federated server with the address "https://mastodon.mit.edu/ https://mastodon.mit.edu/" would consist of those with a tie to the Massachusetts Institute of Technology (faculty or students). Your identity is provided by the system you are a member of and handled by your specific system administrator.
- ilammy 6y ago
  You can fight spam by refusing to federate with servers which are often spamming. That acts as a deterrent for servers to host spam since they'll lose participation. However, this only works if there are enough of other servers to federate with so that you can actually choose your peers. The status quo with email is not like that, unfortunately. Either you federate with GMail and accept the possible spam, or you disrupt significant part of communications of your users by refusing to accept email from Google's mail servers, with Google not losing basically anything (and having no intrinsic reason to fight spam). So it becomes a loss-meh situation instead of win-loss.
  - fragmede 6y ago
    Almost - intrinsic to offering modern webmail, (and not simply selling SMTP as a service) is fighting spam. Empowered users aren't going to put up with receiving spam and can and will go elsewhere.
  - nemothekid 6y ago
    >You can fight spam by refusing to federate with servers which are often spamming Awesome, so users will naturally gravitate to the servers that spent the most $$$ money fighting spam. You end up with Gmail all over again. I don't know why you are using Gmail as the spammer in this situation. Arguably the 2nd biggest reason Gmail got so big is at the time their spam detection was lightyears ahead of everyone else. Then as they got larger they killed off smaller servers that didn't do the magic song and dance, and those users couldn't talk to most people as everyone was on Gmail. There's nothing stopping this from happening again on some new federation platform. This is my biggest gripe with federated services. They all start from the point where "the Internet is centralized" and offer some proto-Internet to undo it. The Internet started federalized and the easiest solution to many of the problems that came with it was centralization. It's always unclear how you prevent this from happening again.
    - wmf 6y ago
      users will naturally gravitate to the servers that spent the most $$$ money fighting spam Or hopefully all servers have good enough open-source anti-spam built in so that it isn't a centralizing force.
- Semiapies 6y ago
  Heh. Honestly, I'd like to see more yammering about federation by people aware of the early federated systems of the internet and some perspective on their uses and failings. Or, better, less yammering and more realizing. Preferably more so than small networks where Nazi Twitter is the biggest node.
- littlestymaar 6y ago
  Email is particularly broken regarding spam for at least two reasons: - it's a distributed system and not really a federated one: you can set up your own SMTP agent and send mails to anyone without prior registration to a federation. - anyone can impersonate anyone: if I want to send an email pretending I'm bill.gates@apple.com there's no built-in mechanism to prevents that. Theses aren't fundamental features of federations, and came from a design which just overlooked how many bad actors email would attract when it became mainstream.
  - remram 6y ago
    You can set up your own Mastodon or Matrix server and send messages, I am not sure which federation you're thinking of that requires "registration". The way the impersonation problem is dealt with in ActivityPub is very similar to DKIM in the SMTP world: the sender uses a cryptographic signature (HTTP signature) to prove that he is allowed to send messages from that actor. Of course this doesn't prevent someone from sending spam from their domain, in either case.
    - TeMPOraL 6y ago
      > I am not sure which federation you're thinking of that requires "registration". Which makes me wonder, how come we call these systems "federated" in the first place? Federations I'm familiar with feature a central authority composed of representatives from the member states. That authority controls admissions and membership, and usually has some level of additional authority over members, along with capability to enforce it. None of that is present in these "federated" systems.
      - wmf 6y ago
        Since many federated systems use DNS, technically ICANN is the central authority. But looking at how the terminology is used in practice, "federated" refers to systems with N users per host while "P2P" describes systems with one user per host.
      - remram 6y ago
- jasonhansel 6y ago
  Spam is a problem on centralized social networks also--and they have advertisements, which are really just well-organized spam. To the extent that this problem is email-specific, the ultimate root cause is a failure to deploy DMARC. Any new federated system will require a DMARC/SPF-like mechanism for preventing request forgery.
  - toast0 6y ago
    DMARC can effectively eliminate email spoofing. Unfortunately, that doesn't prevent spam, phishing, or malware spreading through email. In part because email addresses are largely hidden. In part because not a whole lot of people would notice if their scam mail come from webmaster@somerandomsite.example.net instead of from their bank.
- throwanem 6y ago
  Or a federated system that was designed before anyone imagined spam could be a problem, and thus can only deal with it in a backwards-compatible way by piling on ex post facto hacks. There's no reason to assume the designs of today will be informed by the same naïveté.
erpellan 6y ago
I was slightly surprised to get all the way through the article without seeing a single mention of NNTP.
- hinkley 6y ago
  As a solution or as something that was tried before? Spam as a term transitioned from, “flooding a communication channel with the same message” to “ads sent indiscriminately” due to a green card advertisement on NNTP. What we need along with federation is federated moderation. There are some characterizations of content where we can agree to a label but not whether the label is bad. You might trust me to label gore accurately, I might trust you to label incitement to doxing. We might never agree on what constitutes baiting. Or the crossover point where a flame war should be filtered. So you might ban an individual who I think is “keepin it real” or vice versa. But ten of us shouldn’t have to do the exact same moderation in order to maintain a federation. N log n would be great, 3N would probably be tolerable.
- wmf 6y ago
  In retrospect NNTP and IRC, where every message is replicated to every server, are not a good style of federation.
  - toast0 6y ago
    Does IRC replicate every message to all servers? It's been many years since I was active on IRC, but I thought it was only sending messages along the network tree where needed. So a message to a user would be sent by the sender's server to the recipient's server, or, if not directly connected, to a server one hop closer to that server. And for messages to channels, those only go along the tree if there is a user in the channel from that side of the tree. With today's IRC user count and today's server capabilities, you might have an easier time running on a single server, but there's maybe some latency benefits for users with geographically diverse servers (maybe with support for a server to server mesh instead of a tree, to avoid multi-hop message paths)
asim 6y ago
Strangely I posed a question to HN just a little earlier about the future of sofware - https://news.ycombinator.com/item?id=24547534 https://news.ycombinator.com/item?id=24547534 - and you've answered that question as I was expecting. I have been formulating similar theories and ideas but it's unclear how feasible it is without a collective willing to take it the distance. For me it starts with the basis of a communication protocol for services https://github.com/asim/mucp https://github.com/asim/mucp. And then moves onto something on a broader scale https://gist.github.com/asim/ad6e157294d39c2040a5d51ab3d32168 https://gist.github.com/asim/ad6e157294d39c2040a5d51ab3d3216.... I don't have all the answers yet, but I feel something is coming. An answer to our future needs and its not p2p as you say. P2P might be useful for trade but not services or long term ownership and privacy.
troquerre 6y ago
Traditionally two of the issues with federation have been sharing namespaces and sharing logins. Once you need a single source of truth, the federation breaks and you need to rely on a central authority. There are some new decentralized protocols (Handshake's decentralized naming system is one) that provide a solution to these problems so I'm bullish on federation in general — I think a federated system could break into the mainstream given the recent geopolitical issues affecting centralized services.
- asim 6y ago
  Is it easier to just use federated identity that creates pointers to the origin account? e.g account exists on server A and a new signup to server B creates a link as acc://alice@server-a/blabla
ve55 6y ago
Federated and decentralized solutions have significant UX issues compared to leading products in the area. They're not only harder to use and slower, but are much worse at converting and keeping users to begin with, since they're not ran like businesses are, which take in more funding and try hard to optimize all of these user metrics as KPI. It will be a long time before this can change I think, users still do not (and won't) care if what they're using is federated or not. I say this as someone who loves the idea of federated platforms, but we cannot truly succeed until we have a way to convince large amounts of (normal) users to give them a try.
- ddevault 6y ago
  On the other hand, the fact that no one is thinking about "optimizing KPIs" is a major reason why people like the up and coming federated systems.
  - jcims 6y ago
    Eh, KPIs aren't inherently bad...they just trend that way because of the typical audience for them.
- tylerchilds 6y ago
  I agree about your UX notes, but I disagree with the success criteria. I've wholly quit social media apart from the mastodon instance I'm on. There are enough quality people I'm connected with now that it's a success for me. Am I missing out on content from Facebook/Twitter/Instagram because I'm not connected indirectly to everyone in their gardens? Sure, but I'd argue it's low quality content that was only a drain on my mental health. I guess I'm trying to say, I think it's about the individual success and not the % of market share.
  - ve55 6y ago
    I've enjoyed them myself too, and I still use things like IRC on a daily basis. The two impluses I see to get federation to a larger market share are A) to 'save' people from the platforms they're currently stuck on, but also B) get rid of, or at least diminish, their network effects. You could argue maybe A) won't work or isn't possible or needed, but I feel B) myself a lot. For example I meet someone and want to talk to them, but it's likely they might suggest we chat via Skype or Discord or something like that, and if I don't want to use that service I'm often out of luck. That, personally, is what bothers me the most - that network effects seem inherently anti-competitive because they force me to use a platform I wouldn't otherwise want to use, only because they purposefully make their software unable to communicate with competing platforms. For all of the open protocols and specifications we have from HTTP to TLS to HTML and literally one thousand others, we just re-use them over and over to create closed systems that can't talk to each other, which feels like going backwards. Sure it's great all of our computers speak TCP, UDP, TLS, HTTP, and so on. But they don't all speak iMessage, Zoom, Skype, Discord, Reddit, Twitter, Mastdodon, Matrix, Slack, etc etc. I still feel constantly locked-in to all of these platforms just when I want to communicate to someone that happens to be on it, and the area is certainly not progressing because the fight to gain this huge market share of users is never-ending. Ideally, that is what I would love to end via federation. If I could actually use some type of FOSS and open protocol, and actually talk to any other person I wanted to with it, and I wasn't forced to use a certain platform, service, company, and give them all of my data in the process. I don't see any way for this to be achieved, but I can always dream.
CKN23-ARIN 6y ago
People often forget that "the Internet" itself (i.e. IP, BGP) operates on a federated basis.
dpc_pw 6y ago
Federated systems just don't work in practice because of inherent problems with them that don't have good technical&social solutions yet. Everything on the Internet (including Internet itself) degenerated from federation into monopolistic centralized services for a reason, and somehow people keep ignoring the evidence, and just keep trying harder. These problems are so fundamental and universal that you pretty much need "a new form of Internet" that solves them, so then apps running in it can compete with centralized services. So far the best stab at the problem is Urbit, but mostly because almost nobody else is even trying. Handshake and Holochains are another interesting projects in this space, but more of a pieces of potentially bigger picture.
- IanDrake 6y ago
  You would more likely need a new form of people. Coke is number one for a reason. Sure there is Pepsi, and yes there are small soda makers that either get acquired or close, but not too many people make their own soda. For the same reason, there won't be a federated social network... Unless, people don't even realize they're using one, but that gets back to your point, we'd need a new form of internet.
- zrail 6y ago
  The current shape of the internet itself (BGP peering among autonomous systems, a decent number of large tier 1 ISPs, etc) is, on its face, a relatively successful federation. The applications running atop that federation are, as you say, centralized, but much of that is because residential ISPs forbid customers forbid servers and even if they didn’t (they haven’t always) residential customers are prone to accidentally running botnets of various types.
- Mc91 6y ago
  > Everything on the Internet (including Internet itself) degenerated from federation into monopolistic centralized services for a reason, and somehow people keep ignoring the evidence, and just keep trying harder. Sure there's a reason federated networks like Usenet died - they were killed off by a combination of the old/new AT&T/Verizon duopoly and various government organizations ( https://www.cnet.com/news/n-y-attorney-general-forces-isps-to-curb-usenet-access/ https://www.cnet.com/news/n-y-attorney-general-forces-isps-t... ). Yes, it happened for a reason - the old Bell last-mile monopoly and active government measures to quash it.
jasonhansel 6y ago
For those who dislike federated services, keep in mind: email is a federated system, and it remains arguably one of the largest social networks in the world. Email's federated nature has kept it alive for decades, and will ensure that it long outlives more proprietary forms of social networking. Federated systems already work in practice and at scale. edit: Some other examples with widespread adoption: - The phone network (PSTN) - The internet (via BGP) - The postal service (federated between countries via the UPU) - Podcasts (and RSS more generally) Am I missing any?
- rootusrootus 6y ago
  > arguably one of the largest social networks in the world At the same time, I know several people who gave up their email addresses and only use Facebook now. Maybe the next iteration of social media behemoth will be even more effective at that.
  - walterbell 6y ago
    How do they reset their Facebook password?
    - rootusrootus 6y ago
      Technically they still have an email account, so I imagine that would be how. But they don't otherwise ever use the account, never even check it. Though to be honest I expect they just use their phone if they need to reset their password.
- rtkwe 6y ago
  Unfortunately 2 of those I almost entirely get spam and business/commercial interactions through. On my cell phone I've gotten one call in the last 6 months that wasn't spam and it was an automated notification from the Sheriff about a permit I'd applied for. I haven't gotten a single useful call from someone not on the whitelist of my contact page in so long it's tempting to just autosend all unrecognized numbers to voicemail. Email in particular I don't think really fits the mold of social network. I think principally because there's not really a public post concept it's all essentially point to point communications. There are some abstractions like mailing lists but they're still an express list of people who will get it. I guess that might not be so different from a 100% private profile but the lack of any public functionality excludes it from the social media category.
Mc91 6y ago
I followed somewhat the history of the various file sharing protocols, from Napster and OpenNap to Gnutella to Bittorrent. None of them really stayed totally p2p. With Gnutella, nodes would slow network connections and weird NAT conditions would pop in and pop out. So eventually things changed, and some Gnutella nodes which were online for some time period and had a fast network connection would self-elect as super-nodes (or ultrapeers, or various names). In Gnutella, theoretically ever node was a server and a client, but eventually, Gnutella server/clients would hook into to these super-nodes. If you want the maximum amount of content, both centralization and decentralization have their drawbacks. You tend to find elements of both (with Gnutella and Bittorrent too).
dzink 6y ago
Start this labyrinth from the end point. Why are giants attracting 95%+ of users? 1. Among other things, because of a dead-simple user experience that lets even grandpa post content after a few clicks. Federation, if it adds any level of complexity past the grandpa UX, would defeat the goal. 2. Next, identity. If grandpa can post, so can the bots, how do you ensure accounts don’t go to fake entities? What if an account has to pass a quality threshold before it can publish in public, but can be used privately any time. HN has similar model, but here you limit public posting until 10 random other users have seen and upvoted the new user’s post. Add to that possible additional authentication for payment of a pro feature, or a human check mark, and you’ve put a hurdle on spam. 3. To keep culture from running amuck provide positive and negative feedback. Public content has a span problem, not so much private behavior. Content ratings and discussion licenses can come into play as well, where any group can limit posts to PG content, or discussion that gets moderated automatically or manually if it passes certain thresholds. Happy to suggest more if needed. I run https://www.dreamlist.com https://www.dreamlist.com
- donmcronald 6y ago
  > Next, identity. If grandpa can post, so can the bots, how do you ensure accounts don’t go to fake entities? Invite only. If I'm running a federated service for me and my family, the moderation is going to be astronomically better than anything big providers are doing because a couple minutes of moderation by me every week is astronomically more on a per user basis than what someone like Google or Facebook is doing.
  - chc 6y ago
    That's a nonstarter for a service that's for anyone but the geekiest of geeks, though. If you have to be close to someone with both the tech know-how and interest to host an instance of a federated service, the service is effectively off-limits to approximately everyone. Non-technical users have to be able to use the service and have a good experience doing so.
    - trynewideas 6y ago
shawn-furyan 6y ago
> This system also makes it hard for marketing and spam to get a foothold > You’re certainly familiar with another federation [...]: email
eternalban 6y ago
The only practical issue standing in the way of decentralized services is a lack of business vision. No one has managed or bothered to propose viable business models for decentralized services. Novel approaches to the business side of decentralized services will naturally allow for a re-think of the UX issue. (Business model -> default config for "grandpa") ~ An alternative solution approach requires faith in one's government. Can you trust it? If you can, then a national identity services (yes, the dreaded digital id) can provide the infrastructural back-bone to build federated systems. At the end of the day, location and identity is -all- you need to "reach out and touch someone".
- Icyphox 6y ago
  > No one has managed or bothered to propose viable business models for decentralized services I think that is for the better.
danbolt 6y ago
Earlier in the month, the website OnlyFans set a price cap on what creators could charge for content in response to a celebrity selling photos that weren't as advertised. This affected the incomes of a lot of sex workers whose content contributed to their livelihoods. [1] I'm not knowledgeable about sex work at all, but a lot of the discussion I read noted that it seemed troubling how OnlyFans had so much authority over how the content creators did business. It's not uncommon to hear how sex workers or people in the adult industry are at the whims of opaque moderation on social media too. [2] Sex work can be an exploitative industry, but I think federation could bring a lot of power and control back to the content creators involved online. Disclaimer: Again, I'm not an expert on the topic of sex work or knowledgeable about it at all. It'd be a lot better to get feedback of the people involved or know the industry better. [1] https://www.bbc.com/news/business-53979625 https://www.bbc.com/news/business-53979625 [2] https://jezebel.com/instagram-s-solicitation-policies-are-exposing-porn-per-1834765381 https://jezebel.com/instagram-s-solicitation-policies-are-ex...
- gpm 6y ago
  In the end, the problem sex workers have on the internet is that they are a huge fraud risk, with that risk coming from both the provider and the client. The mainstream payment providers won't touch it as a result. The less mainstream ones are expensive and still not easy to do business with. Without easy setup of payment providers, federation is difficult. Any new instance in the federation needs to go through the same pain. Meanwhile, what does federation actually solve? The only fans model (and most sex work) seems to be a social graph that is a single producer talking to a bunch of clients. The producer needs a server, clients need to connect to that server, where does federation come in? Disclaimer: I'm not at all knowledgeable either - literally just know what I've read on hn on this subject.
  - ampdepolymerase 6y ago
    The bottleneck is the Mastercard/Visa duopoly combined with the modern banking system. At the end of the day no amount of Patreon vs Shopify style business models for adult entertainment work fixes the fundamental payment risk problem. Onlyfans helped amortize the risk somewhat by spreading across multiple payment providers and controlling the UX tightly but even so they still had to change the pricing policy. Mindgeek/CCBill are the Paypal/Stripe of adult entertainment payment processing and it is not an easy field to "disrupt". You can put the best actuaries and machine learning engineers in a room and it would be doubtful if they could come up with a better risk model without fundamentally altering the method of transaction. (Crypto is often touted as a supposed fix for this sort of things but I am not knowledgeable enough about it to comment, somebody else please feel free to chime in.)
laksdjfkasljdf 6y ago
Linux distros/brew/npm/maven/etc are federated software distribution.
- wmf 6y ago
  Not really; most of them have centralized namespaces and servers. PPAs, Flatpak, and Deno are federated though.
  - deleted 6y ago
    [deleted]
ibn_khaldun 6y ago
Mastodon, Misskey and Pleroma are glorified clones of Twitter, and I'm sure there are many more like that. Pixelfed is a glorified clone of Instagram. A few others that I've looked at just don't seem appealing, at least not to me. People need to stop harping over the technical appeal of the Fediverse, lest they only are interested in courting the technical crowd, which is actually fine with me. If that's what they want more of then they should not be concerned with who's a part of it. I think these federated platforms should not be interested in trying to appeal to anyone at all, because it's resulting in imitations of the exact platforms that we want to get away from. Under the hood they may unique, but all most people are going to care about is the front end. Personally, my preference for a new social networking service would be very, very plain. Chat-orientated, share documents, comment on shared documents. No hearts or arrows or "Rebeeps, repoots, and boobops". Is there a way to federate message boards? I just want to talk, read and talk about what I've read. Is there a way to federate Hacker News?
- ernesth 6y ago
  > Is there a way to federate Hacker News? There's Lemmy https://github.com/LemmyNet/lemmy https://github.com/LemmyNet/lemmy
  - mariusor 6y ago
    There are others too.
  - sk2020 6y ago
    I remember talking to the author of Lemmy on Matrix before he made the github repo. Perhaps this has changed, but he didn't seem to have any actual plan for federation. The early work in the repo appeared to be a Vue or Angular frontend (I forget which) with a Reddit-like style with server-to-server communication left as a "TODO". The author seemed to me to be mostly preoccupied with writing essays about why communism is a good idea, so I assumed the author is "an ideas guy" and didn't expect any actual work to get done on the Reddit-clone. Then one day, for no obvious reason, the repo received hundreds of stars within a few hours. It was a trending Rust repo and was recognized on TWiR, but there was no actual implementation of federation at all. Just a JS frontend, and I think a tiny bit of stub code in Rust. I'm not saying there are upvote rings or botnets amongst any particular group, but I haven't paid attention to GitHub stars since then.
shmerl 6y ago
Diaspora* and Matrix are using federation.
Thorentis 6y ago
My issue with federation is that so far, everything still requires some centralisation. Take Matrix for instance. The most popular homeserver by a long, long shot is the one that the Matrix foundation runs themselves. This is same for Mastadon, PeerTube, etc. If you let this experiment play out long enough, you end up with a de facto centralised system. I think P2P ends up being closer to the goal of distributed control because it doesn't tend towards centralisation over time. My issue with P2P though is that it still requires trackers for boostraping in most cases. DHT only gets you so far. I think there will be some interesting advances once IPv6 becomes more widely deployed. Hopefully this time around multicast is widely deployed on the internet. Imagine being able to subscribe to a multicast group and instantly receive DHT nodes over the internet without a tracker? Not sure how this would work in practice, but I have heard of some research being done here. I think the future looks very good for P2P applications, whereas federated applications are still on shaky ground.
- wmf 6y ago
  Ultimately I think developers need to take some responsibility for causing and fixing de facto centralization by not steering new users to "default" servers and refusing signups once a server has excess market share. Federated communication or social media works differently than P2P file sharing. In file sharing you're locating random anonymous peers but in social media that's the last thing that you want. When communicating with a specific person you need their identity anyway and you can use some system like DNS or a blockchain to find the server backing that identity. Multicast on the public Internet is not going to happen due to state management cost, and it's not helpful for federation anyway due to the spam problem.
- Icyphox 6y ago
  > My issue with federation is that so far, everything still requires some centralisation. Take Matrix for instance. The most popular homeserver by a long, long shot is the one that the Matrix foundation runs themselves. This is same for Mastadon, PeerTube, etc. If you let this experiment play out long enough, you end up with a de facto centralised system. Yeah this was Gagron's fault. Running a flagship instance was bound to result in this. I make it a point to educate people about not joining mastodon.social and finding another instance that better suits their interests.
phailhaus 6y ago
I'm not convinced federation is immune to corporatization. The article claims that "it optimizes for a self-governing system of human beings talking to human beings, and not for corporations to push their products." Replace "corporations" with "people", how does federation stop people from using the platform to sell products? Especially when in the same breath, the author admits that operators may be pushed to "solict donations" from their users. Or maybe, ads? Again, you end up with a system in which money is correlated with user experience, and the operators who most efficiently monetize their userbase will be the most successful.
jandrewrogers 6y ago
I work on the design of novel federated systems for adverse environments, like the Internet, where you have to consider the full range of threats to system behavior, functionality, and economy. There is a pretty long list of theoretical trade offs and hard design problems that do not have practical solutions today, which makes it an interesting research area. Some of the more interesting open questions for systems that want to strongly enforce the sovereignty of logical nodes: How do you design inter-node protocol strategies that exhibit optimal, robust Nash equilibria in non-trivial real-world environments i.e. uses resources efficiently in all plausible configurations absent centralized control? This is fiendishly difficult because relevant properties of nodes scale in complex, interacting ways and game theory is not sufficiently developed to solve some of these cases in practical systems. In any sufficiently complex and real-time federated environment, you want these mechanics to be almost completely automagic. Currently, there is a bias toward restricting the number of logical nodes in federated systems because that means humans can plausibly deal with the brittleness, which implies centralization. How do you prevent nodes from being attacked/exploited from both the outside and inside, including malware, DDOS, fraudulent data, exfiltration of private data, etc? This is a separate question from the intrinsic theoretical robustness of the protocol design in terms of resource and scaling efficiency. Also, how do you make a policy easy to describe and enforceable with strong guarantees? The state-of-the-art in high-assurance internetworking (which considers these kinds of problems) is quite far from where it needs to be address these issues convincingly. How do you audit user behavior that spans multiple sovereign federation domains to limit abuse and misuse of the system while leaking minimal private state? Some of these are just Hard Problems, unless you have a very simple/restricted system, that we will need to carefully design around to minimize human involvement; most federated system designs just ignore these issues. For others we have plausible approaches theoretically but no real technical implementations with sufficient generality or functionality. And then there are the fundamental trade offs and edge cases that we tend to ignore but which are really important at scale. tl;dr: federated systems make distributed systems look easy. The reason they are not commonly used is that a correct design for a non-trivial system is deep into the frontier of computer science. We don’t have a complete theory for how to do it.
- Santosh83 6y ago
Santosh83 6y ago
Personally speaking I'd like to see a dead simple to self-host CMS that has built-in comments and RSS, making it trivially easy for people to spin up a simple website or blog or even a webpage, and connect it to a domain they have purchased (or even get one from within the CMS). And by dead simple, I mean something grandpa can do. All they should need is enough skills to install an app, click through a welcome wizard, and start writing on its WYSIWYG editor.
- swiley 6y ago
  You can just open a word processor and save the files to /var/www/ in html format. You don't even need an installation wizard (although on most mainstream OSes you do have to push a button in a GUI somewhere to enable the webserver.) If (for some reason) your computer doesn't have a public address you can install TOR. People with normal browsers will be able to access the pages (even from search engines) via gateways.
  - Santosh83 6y ago
    I get what you're saying but grandpa is not going to be using Unix (Linux) so /var/www/ is out of question. I'm thinking more along the lines of Publii with a built-in Caddy, just to give an example. If the computer doesn't have a publicly reachable IP then that's a much tougher obstacle. That can only be solved by voting with your wallet for ISPs that do give you a public IP with reachable ports. Net neutrality is critical for decentralisation and self-hosting.
    - swiley 6y ago
      This is a feature that works out of the box on OSX. Both /var/www and ~/public are places to put documents. The button to enable appache is in the settings dialog under "sharing" I think. I only use that path because I have no idea what the windows equivalent is but it also ships with a similar feature.
      - Santosh83 6y ago
foreigner 6y ago
Just occurred to me that passports (the paper ones that you use to cross borders) are an example of a federated system.
jondubois 6y ago
Nice article. That said, to talk about the benefits of software federations without conceding that cryptocurrency is the ultimate foundation for such an ecosystem is delusional. Money is the only incentive which works in our modern age. Any federated or decentralized system needs to start with money because money is the ultimate tool for control.
jondubois 6y ago
> Cryptocurrency is the digitally woke techbro’s ponzi scheme The fiat monetary system is a ponzi scheme. Watch "Hidden Secrets of Money": https://youtu.be/iFDe5kUUyT0?t=62 https://youtu.be/iFDe5kUUyT0?t=62 I also recommend doing your own research after this. The evidence is clear and overwhelming. Cryptocurrency is merely a response to this scam and its continued success helped to further prove that the financial system is a scam. Even if you assumed that cryptocurrency is a scam, what kind of financial system enables a scam to thrive for 10+ years? In such a system, how can you tell the scams apart from the non-scams? Clearly the so-called 'free markets' are not capable of distinguishing the two. What's to say that most big tech corporations aren't scams? They get a lot of 'free money' from the government through massive contracts to build projects which only add more bureaucracy.
mariusor 6y ago
This is an eloquent description of what my reasoning was when I started to work on an ActivityPub powered link aggregator[1]. I hope Drew will allow me to re-use some of his words to improve it. [1] https://littr.me/about https://littr.me/about

194 points by als0 6y ago

The biggest challenge I see with federated systems is the discovery process for finding a node you want to be associated with. This is a hard choice - will this server still be running in 6 months? How is it funded? Is it being actively maintained? These questions are too much for laypeople, and I can't imagine a scenario in which your friendly local system admin is your service provider for a critical service in your life like Facebook. Could you imagine email working that way? I see federated systems naturally ending in centralization in the form of a couple big megacorps that the average person trusts to not shut down tomorrow. Convenience and ease of use will win out. It happened with email, it arguably happened with xmpp back in the day. Change my view?

ddevault 6y ago
There's no evidence of this happening with, say, Mastodon. The culture is different, and isn't changing. If we know what we want and what we don't want from the start, we can keep centralization from gaining a foothold.
kixiQu 6y ago
You can migrate a Mastodon account and the old server will forward on your follows.

warkdarrior 6y ago

> "This [federation] system also makes it hard for marketing and spam to get a foothold" > "You’re certainly familiar with another federation which is not based on ActivityPub: email." Self-contradicting article?

ddevault 6y ago
Email is a bit different from the new federations we're seeing today: there's less weaker guarantees of identity, and there's less emphasis on choosing what other instances you federate with. In the example of Mastodon, the cost of setting up a new server is also too high to make spam effective, since you need a persistent, reachable account in order to interact with other instances on behalf of it. And the social factor counts, too, newer federations like this tend to be opposed to corporate influence in general and this makes it hard for it to get the initial foothold. Email has a lot of spam, but a federated system which keeps this in mind from the outset needn't suffer the same fate. EDIT: Incorporating the replies to my comment, I also want to mention that the lack of large, centralized players like gmail also prevents spam. If there are a lot of intances, and they're all small, then cutting one off for bad behavior is a lot less difficult. Instance admins can be held accountable for the behavior of their users.
- Jasper_ 6y ago
  That would imply that centralized solutions would have even less spam, since they have much stronger guarantees of identity. And yet sites like Twitter has plenty of spam. Even Hacker News gets spam! And this is not a popular place in the grand scheme of things. How is distributed identity solved in the long term? Nobody would run a server to communicate spam; that's rarely how email spam is sent these days. Most of the spam I get is from accounts hosted on GMail and Yahoo!. So the weakest link is a single server that's community-respected, but doesn't have adequate spam protection. Federation makes that much harder, because the whole point is that every server is run separately. Mastodon gets no spam today because it's smaller than Hacker News. There's relatively nobody on there.
  - kzrdude 6y ago
    I'm worried how mastodon is going to scale (in terms of moderation) - maybe some instance can experient with crowd sourced moderation.
    - indigochill 6y ago
      Part of the point, as I understand it, is that moderation scales more-or-less linearly because each instance is ideally human-scale (that is, maybe having hundreds of members as opposed to millions) and responsible for its own moderation, unlike sites like Twitter or Facebook which have to pay a relatively small team of moderators to moderate millions of people's communications. There's nothing saying any particular instance will moderate, but if they don't and it becomes a problem they'll quickly find themselves cut off from federating with other instances.
  - riffic 6y ago
    > How is distributed identity solved in the long term? Same way email handles it -- through the use of the domain name system. You'd expect that an a federated server with the address "https://mastodon.mit.edu/ https://mastodon.mit.edu/" would consist of those with a tie to the Massachusetts Institute of Technology (faculty or students). Your identity is provided by the system you are a member of and handled by your specific system administrator.
ilammy 6y ago
You can fight spam by refusing to federate with servers which are often spamming. That acts as a deterrent for servers to host spam since they'll lose participation. However, this only works if there are enough of other servers to federate with so that you can actually choose your peers. The status quo with email is not like that, unfortunately. Either you federate with GMail and accept the possible spam, or you disrupt significant part of communications of your users by refusing to accept email from Google's mail servers, with Google not losing basically anything (and having no intrinsic reason to fight spam). So it becomes a loss-meh situation instead of win-loss.
- fragmede 6y ago
  Almost - intrinsic to offering modern webmail, (and not simply selling SMTP as a service) is fighting spam. Empowered users aren't going to put up with receiving spam and can and will go elsewhere.
- nemothekid 6y ago
  >You can fight spam by refusing to federate with servers which are often spamming Awesome, so users will naturally gravitate to the servers that spent the most $$$ money fighting spam. You end up with Gmail all over again. I don't know why you are using Gmail as the spammer in this situation. Arguably the 2nd biggest reason Gmail got so big is at the time their spam detection was lightyears ahead of everyone else. Then as they got larger they killed off smaller servers that didn't do the magic song and dance, and those users couldn't talk to most people as everyone was on Gmail. There's nothing stopping this from happening again on some new federation platform. This is my biggest gripe with federated services. They all start from the point where "the Internet is centralized" and offer some proto-Internet to undo it. The Internet started federalized and the easiest solution to many of the problems that came with it was centralization. It's always unclear how you prevent this from happening again.
  - wmf 6y ago
    users will naturally gravitate to the servers that spent the most $$$ money fighting spam Or hopefully all servers have good enough open-source anti-spam built in so that it isn't a centralizing force.
Semiapies 6y ago
Heh. Honestly, I'd like to see more yammering about federation by people aware of the early federated systems of the internet and some perspective on their uses and failings. Or, better, less yammering and more realizing. Preferably more so than small networks where Nazi Twitter is the biggest node.
littlestymaar 6y ago
Email is particularly broken regarding spam for at least two reasons: - it's a distributed system and not really a federated one: you can set up your own SMTP agent and send mails to anyone without prior registration to a federation. - anyone can impersonate anyone: if I want to send an email pretending I'm bill.gates@apple.com there's no built-in mechanism to prevents that. Theses aren't fundamental features of federations, and came from a design which just overlooked how many bad actors email would attract when it became mainstream.
- remram 6y ago
  You can set up your own Mastodon or Matrix server and send messages, I am not sure which federation you're thinking of that requires "registration". The way the impersonation problem is dealt with in ActivityPub is very similar to DKIM in the SMTP world: the sender uses a cryptographic signature (HTTP signature) to prove that he is allowed to send messages from that actor. Of course this doesn't prevent someone from sending spam from their domain, in either case.
  - TeMPOraL 6y ago
    > I am not sure which federation you're thinking of that requires "registration". Which makes me wonder, how come we call these systems "federated" in the first place? Federations I'm familiar with feature a central authority composed of representatives from the member states. That authority controls admissions and membership, and usually has some level of additional authority over members, along with capability to enforce it. None of that is present in these "federated" systems.
    - wmf 6y ago
      Since many federated systems use DNS, technically ICANN is the central authority. But looking at how the terminology is used in practice, "federated" refers to systems with N users per host while "P2P" describes systems with one user per host.
    - remram 6y ago
jasonhansel 6y ago
Spam is a problem on centralized social networks also--and they have advertisements, which are really just well-organized spam. To the extent that this problem is email-specific, the ultimate root cause is a failure to deploy DMARC. Any new federated system will require a DMARC/SPF-like mechanism for preventing request forgery.
- toast0 6y ago
  DMARC can effectively eliminate email spoofing. Unfortunately, that doesn't prevent spam, phishing, or malware spreading through email. In part because email addresses are largely hidden. In part because not a whole lot of people would notice if their scam mail come from webmaster@somerandomsite.example.net instead of from their bank.
throwanem 6y ago
Or a federated system that was designed before anyone imagined spam could be a problem, and thus can only deal with it in a backwards-compatible way by piling on ex post facto hacks. There's no reason to assume the designs of today will be informed by the same naïveté.

erpellan 6y ago

I was slightly surprised to get all the way through the article without seeing a single mention of NNTP.

hinkley 6y ago
As a solution or as something that was tried before? Spam as a term transitioned from, “flooding a communication channel with the same message” to “ads sent indiscriminately” due to a green card advertisement on NNTP. What we need along with federation is federated moderation. There are some characterizations of content where we can agree to a label but not whether the label is bad. You might trust me to label gore accurately, I might trust you to label incitement to doxing. We might never agree on what constitutes baiting. Or the crossover point where a flame war should be filtered. So you might ban an individual who I think is “keepin it real” or vice versa. But ten of us shouldn’t have to do the exact same moderation in order to maintain a federation. N log n would be great, 3N would probably be tolerable.
wmf 6y ago
In retrospect NNTP and IRC, where every message is replicated to every server, are not a good style of federation.
- toast0 6y ago
  Does IRC replicate every message to all servers? It's been many years since I was active on IRC, but I thought it was only sending messages along the network tree where needed. So a message to a user would be sent by the sender's server to the recipient's server, or, if not directly connected, to a server one hop closer to that server. And for messages to channels, those only go along the tree if there is a user in the channel from that side of the tree. With today's IRC user count and today's server capabilities, you might have an easier time running on a single server, but there's maybe some latency benefits for users with geographically diverse servers (maybe with support for a server to server mesh instead of a tree, to avoid multi-hop message paths)

asim 6y ago

Strangely I posed a question to HN just a little earlier about the future of sofware - https://news.ycombinator.com/item?id=24547534 https://news.ycombinator.com/item?id=24547534 - and you've answered that question as I was expecting. I have been formulating similar theories and ideas but it's unclear how feasible it is without a collective willing to take it the distance. For me it starts with the basis of a communication protocol for services https://github.com/asim/mucp https://github.com/asim/mucp. And then moves onto something on a broader scale https://gist.github.com/asim/ad6e157294d39c2040a5d51ab3d32168 https://gist.github.com/asim/ad6e157294d39c2040a5d51ab3d3216.... I don't have all the answers yet, but I feel something is coming. An answer to our future needs and its not p2p as you say. P2P might be useful for trade but not services or long term ownership and privacy.

troquerre 6y ago

Traditionally two of the issues with federation have been sharing namespaces and sharing logins. Once you need a single source of truth, the federation breaks and you need to rely on a central authority. There are some new decentralized protocols (Handshake's decentralized naming system is one) that provide a solution to these problems so I'm bullish on federation in general — I think a federated system could break into the mainstream given the recent geopolitical issues affecting centralized services.

asim 6y ago
Is it easier to just use federated identity that creates pointers to the origin account? e.g account exists on server A and a new signup to server B creates a link as acc://alice@server-a/blabla

ve55 6y ago

Federated and decentralized solutions have significant UX issues compared to leading products in the area. They're not only harder to use and slower, but are much worse at converting and keeping users to begin with, since they're not ran like businesses are, which take in more funding and try hard to optimize all of these user metrics as KPI. It will be a long time before this can change I think, users still do not (and won't) care if what they're using is federated or not. I say this as someone who loves the idea of federated platforms, but we cannot truly succeed until we have a way to convince large amounts of (normal) users to give them a try.

ddevault 6y ago
On the other hand, the fact that no one is thinking about "optimizing KPIs" is a major reason why people like the up and coming federated systems.
- jcims 6y ago
  Eh, KPIs aren't inherently bad...they just trend that way because of the typical audience for them.
tylerchilds 6y ago
I agree about your UX notes, but I disagree with the success criteria. I've wholly quit social media apart from the mastodon instance I'm on. There are enough quality people I'm connected with now that it's a success for me. Am I missing out on content from Facebook/Twitter/Instagram because I'm not connected indirectly to everyone in their gardens? Sure, but I'd argue it's low quality content that was only a drain on my mental health. I guess I'm trying to say, I think it's about the individual success and not the % of market share.
- ve55 6y ago
  I've enjoyed them myself too, and I still use things like IRC on a daily basis. The two impluses I see to get federation to a larger market share are A) to 'save' people from the platforms they're currently stuck on, but also B) get rid of, or at least diminish, their network effects. You could argue maybe A) won't work or isn't possible or needed, but I feel B) myself a lot. For example I meet someone and want to talk to them, but it's likely they might suggest we chat via Skype or Discord or something like that, and if I don't want to use that service I'm often out of luck. That, personally, is what bothers me the most - that network effects seem inherently anti-competitive because they force me to use a platform I wouldn't otherwise want to use, only because they purposefully make their software unable to communicate with competing platforms. For all of the open protocols and specifications we have from HTTP to TLS to HTML and literally one thousand others, we just re-use them over and over to create closed systems that can't talk to each other, which feels like going backwards. Sure it's great all of our computers speak TCP, UDP, TLS, HTTP, and so on. But they don't all speak iMessage, Zoom, Skype, Discord, Reddit, Twitter, Mastdodon, Matrix, Slack, etc etc. I still feel constantly locked-in to all of these platforms just when I want to communicate to someone that happens to be on it, and the area is certainly not progressing because the fight to gain this huge market share of users is never-ending. Ideally, that is what I would love to end via federation. If I could actually use some type of FOSS and open protocol, and actually talk to any other person I wanted to with it, and I wasn't forced to use a certain platform, service, company, and give them all of my data in the process. I don't see any way for this to be achieved, but I can always dream.

CKN23-ARIN 6y ago

People often forget that "the Internet" itself (i.e. IP, BGP) operates on a federated basis.

dpc_pw 6y ago

Federated systems just don't work in practice because of inherent problems with them that don't have good technical&social solutions yet. Everything on the Internet (including Internet itself) degenerated from federation into monopolistic centralized services for a reason, and somehow people keep ignoring the evidence, and just keep trying harder. These problems are so fundamental and universal that you pretty much need "a new form of Internet" that solves them, so then apps running in it can compete with centralized services. So far the best stab at the problem is Urbit, but mostly because almost nobody else is even trying. Handshake and Holochains are another interesting projects in this space, but more of a pieces of potentially bigger picture.

IanDrake 6y ago
You would more likely need a new form of people. Coke is number one for a reason. Sure there is Pepsi, and yes there are small soda makers that either get acquired or close, but not too many people make their own soda. For the same reason, there won't be a federated social network... Unless, people don't even realize they're using one, but that gets back to your point, we'd need a new form of internet.
zrail 6y ago
The current shape of the internet itself (BGP peering among autonomous systems, a decent number of large tier 1 ISPs, etc) is, on its face, a relatively successful federation. The applications running atop that federation are, as you say, centralized, but much of that is because residential ISPs forbid customers forbid servers and even if they didn’t (they haven’t always) residential customers are prone to accidentally running botnets of various types.
Mc91 6y ago
> Everything on the Internet (including Internet itself) degenerated from federation into monopolistic centralized services for a reason, and somehow people keep ignoring the evidence, and just keep trying harder. Sure there's a reason federated networks like Usenet died - they were killed off by a combination of the old/new AT&T/Verizon duopoly and various government organizations ( https://www.cnet.com/news/n-y-attorney-general-forces-isps-to-curb-usenet-access/ https://www.cnet.com/news/n-y-attorney-general-forces-isps-t... ). Yes, it happened for a reason - the old Bell last-mile monopoly and active government measures to quash it.

jasonhansel 6y ago

For those who dislike federated services, keep in mind: email is a federated system, and it remains arguably one of the largest social networks in the world. Email's federated nature has kept it alive for decades, and will ensure that it long outlives more proprietary forms of social networking. Federated systems already work in practice and at scale. edit: Some other examples with widespread adoption: - The phone network (PSTN) - The internet (via BGP) - The postal service (federated between countries via the UPU) - Podcasts (and RSS more generally) Am I missing any?

rootusrootus 6y ago
> arguably one of the largest social networks in the world At the same time, I know several people who gave up their email addresses and only use Facebook now. Maybe the next iteration of social media behemoth will be even more effective at that.
- walterbell 6y ago
  How do they reset their Facebook password?
  - rootusrootus 6y ago
    Technically they still have an email account, so I imagine that would be how. But they don't otherwise ever use the account, never even check it. Though to be honest I expect they just use their phone if they need to reset their password.
rtkwe 6y ago
Unfortunately 2 of those I almost entirely get spam and business/commercial interactions through. On my cell phone I've gotten one call in the last 6 months that wasn't spam and it was an automated notification from the Sheriff about a permit I'd applied for. I haven't gotten a single useful call from someone not on the whitelist of my contact page in so long it's tempting to just autosend all unrecognized numbers to voicemail. Email in particular I don't think really fits the mold of social network. I think principally because there's not really a public post concept it's all essentially point to point communications. There are some abstractions like mailing lists but they're still an express list of people who will get it. I guess that might not be so different from a 100% private profile but the lack of any public functionality excludes it from the social media category.

Mc91 6y ago

I followed somewhat the history of the various file sharing protocols, from Napster and OpenNap to Gnutella to Bittorrent. None of them really stayed totally p2p. With Gnutella, nodes would slow network connections and weird NAT conditions would pop in and pop out. So eventually things changed, and some Gnutella nodes which were online for some time period and had a fast network connection would self-elect as super-nodes (or ultrapeers, or various names). In Gnutella, theoretically ever node was a server and a client, but eventually, Gnutella server/clients would hook into to these super-nodes. If you want the maximum amount of content, both centralization and decentralization have their drawbacks. You tend to find elements of both (with Gnutella and Bittorrent too).

dzink 6y ago

Start this labyrinth from the end point. Why are giants attracting 95%+ of users? 1. Among other things, because of a dead-simple user experience that lets even grandpa post content after a few clicks. Federation, if it adds any level of complexity past the grandpa UX, would defeat the goal. 2. Next, identity. If grandpa can post, so can the bots, how do you ensure accounts don’t go to fake entities? What if an account has to pass a quality threshold before it can publish in public, but can be used privately any time. HN has similar model, but here you limit public posting until 10 random other users have seen and upvoted the new user’s post. Add to that possible additional authentication for payment of a pro feature, or a human check mark, and you’ve put a hurdle on spam. 3. To keep culture from running amuck provide positive and negative feedback. Public content has a span problem, not so much private behavior. Content ratings and discussion licenses can come into play as well, where any group can limit posts to PG content, or discussion that gets moderated automatically or manually if it passes certain thresholds. Happy to suggest more if needed. I run https://www.dreamlist.com https://www.dreamlist.com

donmcronald 6y ago
> Next, identity. If grandpa can post, so can the bots, how do you ensure accounts don’t go to fake entities? Invite only. If I'm running a federated service for me and my family, the moderation is going to be astronomically better than anything big providers are doing because a couple minutes of moderation by me every week is astronomically more on a per user basis than what someone like Google or Facebook is doing.
- chc 6y ago
  That's a nonstarter for a service that's for anyone but the geekiest of geeks, though. If you have to be close to someone with both the tech know-how and interest to host an instance of a federated service, the service is effectively off-limits to approximately everyone. Non-technical users have to be able to use the service and have a good experience doing so.
  - trynewideas 6y ago

shawn-furyan 6y ago

> This system also makes it hard for marketing and spam to get a foothold > You’re certainly familiar with another federation [...]: email

eternalban 6y ago

The only practical issue standing in the way of decentralized services is a lack of business vision. No one has managed or bothered to propose viable business models for decentralized services. Novel approaches to the business side of decentralized services will naturally allow for a re-think of the UX issue. (Business model -> default config for "grandpa") ~ An alternative solution approach requires faith in one's government. Can you trust it? If you can, then a national identity services (yes, the dreaded digital id) can provide the infrastructural back-bone to build federated systems. At the end of the day, location and identity is -all- you need to "reach out and touch someone".

Icyphox 6y ago
> No one has managed or bothered to propose viable business models for decentralized services I think that is for the better.

danbolt 6y ago

Earlier in the month, the website OnlyFans set a price cap on what creators could charge for content in response to a celebrity selling photos that weren't as advertised. This affected the incomes of a lot of sex workers whose content contributed to their livelihoods. [1] I'm not knowledgeable about sex work at all, but a lot of the discussion I read noted that it seemed troubling how OnlyFans had so much authority over how the content creators did business. It's not uncommon to hear how sex workers or people in the adult industry are at the whims of opaque moderation on social media too. [2] Sex work can be an exploitative industry, but I think federation could bring a lot of power and control back to the content creators involved online. Disclaimer: Again, I'm not an expert on the topic of sex work or knowledgeable about it at all. It'd be a lot better to get feedback of the people involved or know the industry better. [1] https://www.bbc.com/news/business-53979625 https://www.bbc.com/news/business-53979625 [2] https://jezebel.com/instagram-s-solicitation-policies-are-exposing-porn-per-1834765381 https://jezebel.com/instagram-s-solicitation-policies-are-ex...

gpm 6y ago
In the end, the problem sex workers have on the internet is that they are a huge fraud risk, with that risk coming from both the provider and the client. The mainstream payment providers won't touch it as a result. The less mainstream ones are expensive and still not easy to do business with. Without easy setup of payment providers, federation is difficult. Any new instance in the federation needs to go through the same pain. Meanwhile, what does federation actually solve? The only fans model (and most sex work) seems to be a social graph that is a single producer talking to a bunch of clients. The producer needs a server, clients need to connect to that server, where does federation come in? Disclaimer: I'm not at all knowledgeable either - literally just know what I've read on hn on this subject.
- ampdepolymerase 6y ago
  The bottleneck is the Mastercard/Visa duopoly combined with the modern banking system. At the end of the day no amount of Patreon vs Shopify style business models for adult entertainment work fixes the fundamental payment risk problem. Onlyfans helped amortize the risk somewhat by spreading across multiple payment providers and controlling the UX tightly but even so they still had to change the pricing policy. Mindgeek/CCBill are the Paypal/Stripe of adult entertainment payment processing and it is not an easy field to "disrupt". You can put the best actuaries and machine learning engineers in a room and it would be doubtful if they could come up with a better risk model without fundamentally altering the method of transaction. (Crypto is often touted as a supposed fix for this sort of things but I am not knowledgeable enough about it to comment, somebody else please feel free to chime in.)

laksdjfkasljdf 6y ago

Linux distros/brew/npm/maven/etc are federated software distribution.

wmf 6y ago
Not really; most of them have centralized namespaces and servers. PPAs, Flatpak, and Deno are federated though.
- deleted 6y ago
  [deleted]

ibn_khaldun 6y ago

Mastodon, Misskey and Pleroma are glorified clones of Twitter, and I'm sure there are many more like that. Pixelfed is a glorified clone of Instagram. A few others that I've looked at just don't seem appealing, at least not to me. People need to stop harping over the technical appeal of the Fediverse, lest they only are interested in courting the technical crowd, which is actually fine with me. If that's what they want more of then they should not be concerned with who's a part of it. I think these federated platforms should not be interested in trying to appeal to anyone at all, because it's resulting in imitations of the exact platforms that we want to get away from. Under the hood they may unique, but all most people are going to care about is the front end. Personally, my preference for a new social networking service would be very, very plain. Chat-orientated, share documents, comment on shared documents. No hearts or arrows or "Rebeeps, repoots, and boobops". Is there a way to federate message boards? I just want to talk, read and talk about what I've read. Is there a way to federate Hacker News?

ernesth 6y ago
> Is there a way to federate Hacker News? There's Lemmy https://github.com/LemmyNet/lemmy https://github.com/LemmyNet/lemmy
- mariusor 6y ago
  There are others too.
- sk2020 6y ago
  I remember talking to the author of Lemmy on Matrix before he made the github repo. Perhaps this has changed, but he didn't seem to have any actual plan for federation. The early work in the repo appeared to be a Vue or Angular frontend (I forget which) with a Reddit-like style with server-to-server communication left as a "TODO". The author seemed to me to be mostly preoccupied with writing essays about why communism is a good idea, so I assumed the author is "an ideas guy" and didn't expect any actual work to get done on the Reddit-clone. Then one day, for no obvious reason, the repo received hundreds of stars within a few hours. It was a trending Rust repo and was recognized on TWiR, but there was no actual implementation of federation at all. Just a JS frontend, and I think a tiny bit of stub code in Rust. I'm not saying there are upvote rings or botnets amongst any particular group, but I haven't paid attention to GitHub stars since then.

shmerl 6y ago

Diaspora* and Matrix are using federation.

Thorentis 6y ago

My issue with federation is that so far, everything still requires some centralisation. Take Matrix for instance. The most popular homeserver by a long, long shot is the one that the Matrix foundation runs themselves. This is same for Mastadon, PeerTube, etc. If you let this experiment play out long enough, you end up with a de facto centralised system. I think P2P ends up being closer to the goal of distributed control because it doesn't tend towards centralisation over time. My issue with P2P though is that it still requires trackers for boostraping in most cases. DHT only gets you so far. I think there will be some interesting advances once IPv6 becomes more widely deployed. Hopefully this time around multicast is widely deployed on the internet. Imagine being able to subscribe to a multicast group and instantly receive DHT nodes over the internet without a tracker? Not sure how this would work in practice, but I have heard of some research being done here. I think the future looks very good for P2P applications, whereas federated applications are still on shaky ground.

wmf 6y ago
Ultimately I think developers need to take some responsibility for causing and fixing de facto centralization by not steering new users to "default" servers and refusing signups once a server has excess market share. Federated communication or social media works differently than P2P file sharing. In file sharing you're locating random anonymous peers but in social media that's the last thing that you want. When communicating with a specific person you need their identity anyway and you can use some system like DNS or a blockchain to find the server backing that identity. Multicast on the public Internet is not going to happen due to state management cost, and it's not helpful for federation anyway due to the spam problem.
Icyphox 6y ago
> My issue with federation is that so far, everything still requires some centralisation. Take Matrix for instance. The most popular homeserver by a long, long shot is the one that the Matrix foundation runs themselves. This is same for Mastadon, PeerTube, etc. If you let this experiment play out long enough, you end up with a de facto centralised system. Yeah this was Gagron's fault. Running a flagship instance was bound to result in this. I make it a point to educate people about not joining mastodon.social and finding another instance that better suits their interests.

phailhaus 6y ago

I'm not convinced federation is immune to corporatization. The article claims that "it optimizes for a self-governing system of human beings talking to human beings, and not for corporations to push their products." Replace "corporations" with "people", how does federation stop people from using the platform to sell products? Especially when in the same breath, the author admits that operators may be pushed to "solict donations" from their users. Or maybe, ads? Again, you end up with a system in which money is correlated with user experience, and the operators who most efficiently monetize their userbase will be the most successful.

jandrewrogers 6y ago

I work on the design of novel federated systems for adverse environments, like the Internet, where you have to consider the full range of threats to system behavior, functionality, and economy. There is a pretty long list of theoretical trade offs and hard design problems that do not have practical solutions today, which makes it an interesting research area. Some of the more interesting open questions for systems that want to strongly enforce the sovereignty of logical nodes: How do you design inter-node protocol strategies that exhibit optimal, robust Nash equilibria in non-trivial real-world environments i.e. uses resources efficiently in all plausible configurations absent centralized control? This is fiendishly difficult because relevant properties of nodes scale in complex, interacting ways and game theory is not sufficiently developed to solve some of these cases in practical systems. In any sufficiently complex and real-time federated environment, you want these mechanics to be almost completely automagic. Currently, there is a bias toward restricting the number of logical nodes in federated systems because that means humans can plausibly deal with the brittleness, which implies centralization. How do you prevent nodes from being attacked/exploited from both the outside and inside, including malware, DDOS, fraudulent data, exfiltration of private data, etc? This is a separate question from the intrinsic theoretical robustness of the protocol design in terms of resource and scaling efficiency. Also, how do you make a policy easy to describe and enforceable with strong guarantees? The state-of-the-art in high-assurance internetworking (which considers these kinds of problems) is quite far from where it needs to be address these issues convincingly. How do you audit user behavior that spans multiple sovereign federation domains to limit abuse and misuse of the system while leaking minimal private state? Some of these are just Hard Problems, unless you have a very simple/restricted system, that we will need to carefully design around to minimize human involvement; most federated system designs just ignore these issues. For others we have plausible approaches theoretically but no real technical implementations with sufficient generality or functionality. And then there are the fundamental trade offs and edge cases that we tend to ignore but which are really important at scale. tl;dr: federated systems make distributed systems look easy. The reason they are not commonly used is that a correct design for a non-trivial system is deep into the frontier of computer science. We don’t have a complete theory for how to do it.

Santosh83 6y ago

Santosh83 6y ago

Personally speaking I'd like to see a dead simple to self-host CMS that has built-in comments and RSS, making it trivially easy for people to spin up a simple website or blog or even a webpage, and connect it to a domain they have purchased (or even get one from within the CMS). And by dead simple, I mean something grandpa can do. All they should need is enough skills to install an app, click through a welcome wizard, and start writing on its WYSIWYG editor.

swiley 6y ago
You can just open a word processor and save the files to /var/www/ in html format. You don't even need an installation wizard (although on most mainstream OSes you do have to push a button in a GUI somewhere to enable the webserver.) If (for some reason) your computer doesn't have a public address you can install TOR. People with normal browsers will be able to access the pages (even from search engines) via gateways.
- Santosh83 6y ago
  I get what you're saying but grandpa is not going to be using Unix (Linux) so /var/www/ is out of question. I'm thinking more along the lines of Publii with a built-in Caddy, just to give an example. If the computer doesn't have a publicly reachable IP then that's a much tougher obstacle. That can only be solved by voting with your wallet for ISPs that do give you a public IP with reachable ports. Net neutrality is critical for decentralisation and self-hosting.
  - swiley 6y ago
    This is a feature that works out of the box on OSX. Both /var/www and ~/public are places to put documents. The button to enable appache is in the settings dialog under "sharing" I think. I only use that path because I have no idea what the windows equivalent is but it also ships with a similar feature.
    - Santosh83 6y ago

foreigner 6y ago

Just occurred to me that passports (the paper ones that you use to cross borders) are an example of a federated system.

jondubois 6y ago

Nice article. That said, to talk about the benefits of software federations without conceding that cryptocurrency is the ultimate foundation for such an ecosystem is delusional. Money is the only incentive which works in our modern age. Any federated or decentralized system needs to start with money because money is the ultimate tool for control.

jondubois 6y ago

> Cryptocurrency is the digitally woke techbro’s ponzi scheme The fiat monetary system is a ponzi scheme. Watch "Hidden Secrets of Money": https://youtu.be/iFDe5kUUyT0?t=62 https://youtu.be/iFDe5kUUyT0?t=62 I also recommend doing your own research after this. The evidence is clear and overwhelming. Cryptocurrency is merely a response to this scam and its continued success helped to further prove that the financial system is a scam. Even if you assumed that cryptocurrency is a scam, what kind of financial system enables a scam to thrive for 10+ years? In such a system, how can you tell the scams apart from the non-scams? Clearly the so-called 'free markets' are not capable of distinguishing the two. What's to say that most big tech corporations aren't scams? They get a lot of 'free money' from the government through massive contracts to build projects which only add more bureaucracy.

mariusor 6y ago

This is an eloquent description of what my reasoning was when I started to work on an ActivityPub powered link aggregator[1]. I hope Drew will allow me to re-use some of his words to improve it. [1] https://littr.me/about https://littr.me/about