26 ms·

Scaling Mastodon is impossible

• barbariangrunge 4y ago
  > I have a lot of thoughts on this that are too long for a Tweet or Toot. Since some of my followers asked though I decided do a longform version of this People should leave Twitter and start blogs so that they can talk about more complex things with more nuance
  • poulpy123 4y ago
    Maybe we should call that web 4.0
    • RamblingCTO 4y ago
      Web 2 2.0
  • pwinnski 4y ago
    Some mastodon instances allow for much longer than 500-character posts. The qoto.org mentioned in the article allows 64K, perfect for blogging.
    • plgonzalezrx8 4y ago
      infosec.exchange allows up to 11000.
  • bombcar 4y ago
    What you need, really, is centralized "big squares" where people can talk about limited topics with strong oversight (think: HN) but they can link too off-site spaces where smaller groups can have their own rules and topics. I know some of these types of groups, and I don't really publicize them, because the people who would fit in will find them, no need to grow to world-wide size.
• BeefWellington 4y ago
  It would be great if the author backed up, really, any of their points with some kind of demonstration about what they're talking about. This article otherwise is seemingly a bunch of handwaving about vague "problems" that exist but without actually clearly stating examples. Also the author is flat out incorrect on this: > This decentralization however came with a lot of challenges and today decentralized package hosting is no longer supported by the Python ecosystem. pip has supported github urls for a long time. edit Further: > The second thing that became apparent over time was also that decentralized services came with a lot of security risks. Every one of those hosts allowed the re-publishing of already existing packages. Domains that lapsed could be re-registered by other people and new packages could be placed there. Linux package management systems solved the decentralization "problem" years ago. It's why there's so many mirrors available when you download packages. Signed packages, and even "trusted-source" checksums can provide for integrity in cases where decentralization exists. That some package managers threw this away in favour of monolithic repositories is kind of irrelevant to how ActivityPub works.
  • the_mitsuhiko 4y ago
    Author here. > pip has supported github urls for a long time. setuptools used to be able to pull packages published to the index from external URLs. Support for this was removed many years ago. Also you cannot publish packages to PyPI that reference dependencies on GitHub. About the rest of your points on package indexes I believe I addressed them in the post already.
    • BeefWellington 4y ago
      That's specific to one scenario for Python. It's hardly the entire ecosystem, which is my point. Decentralization is there already and well-supported in the provided tools. You just can't abuse the official PyPI sources, and that's a reasonable approach.
      • the_mitsuhiko 4y ago
        The point I was making is that systems have a tendency to centralize, even in the absence of a bad actor. Decentralization was really well supported in the Python ecosystem in former times, it no longer is. The trend is pretty self evident I would argue.
• LinuxBender 4y ago
  The article goes into some details beyond scaling that I can resonate with. I had a few forums and IRC servers in the past that grew rather large. I eventually shut them down, not because of scalability but because of legal liability and dealing with the myriad of personality issues that put my domains at risk. Scaling a forum or IRC to hundreds of thousands or even millions of people is not hard especially nowadays with cloud scaling and the current state of modern kernels and hardware. What I found too challenging was having to moderate the content and finding moderators that could be trusted to remove illegal content in a timely manor. Worse, there were trolls that would use bots to post highly illegal material and then automatically submit their own posts to my registrars, server providers and government. The bots somehow even grabbed screenshots right after they posted content. I say bots because there was no way a human to perform their actions so quickly. This was a losing battle and I did not have the legal resources to deal with it, nor the development resources to play the cat and mouse arms race 24/7. I do have my own conspiracy theories as to who these bot owners were but that doesn't matter any more. Nowadays I could probably block more of those bots with techniques I have learned but I just do not have the desire to get back into that quagmire. I suspect some of the Mastodon admins will learn this lesson with time. They, like me, will probably start in a state of denial and dismiss the risk until it gets real. And it certainly gets real. The only technical work around I could find was to set forums to make all posts moderator-approved, meaning only the poster can see their post until a moderator approves it. This does not scale and people want their posts to be instantly available. With IRC I had to constantly add new file sharing domains to word filters to block the links to illegal material and that was also a losing battle. [Edit] BeefWellington brings up a good point. I should add that I am referring to public instances of forums and IRC servers that anyone may join. Private servers are at much lower risk assuming the trusted members are good at setting strong passwords and static content is not accessible at all without an account and Mastodon servers are not linked to lesser trusted or non-private instances.
  • BeefWellington 4y ago
    > I suspect some of the Mastodon admins will learn this lesson with time. I doubt it. A lot of the new instances are invitation-only, and the point of Federation is I can just run my own instance and seek out the content I desire. I don't have to let anyone else onto my instance.
• schwartzworld 4y ago
  > My Claim: Decentralization is a Questionable Goal Hard disagree with this. Email is the example I give people of how federation could work. You can use any email provider and interact with any other email provider effortlessly. This is undeniably an improvement over internal direct messages within a centralized service (i.e. Facebook or Twitter DMs). If you leave facebook, you leave your contact list and message history. On email, if using a custom domain, you can switch email providers or even host your own server without anybody on your contact list even having to know this occurred. Lots of people have been banned from platforms, rightly or wrongly. You can't get banned from email. You can't get banned from having your own website. You can't get banned from the telephone system. If you don't own your own data, you are at the mercy of those who do. > On the topic of moderation the very same issue is even more absurd. Some instances want uncontrolled free speech where some instances effectively are pure shit-posting instances which are completely de-federated from the most of the fediverse as a result. Other instances really like to control their content, where some popular ones such as fosstodon ban all languages than English as a result to allow moderation. These are talked about as problems, rather than being the primary selling point.
  • humanistbot 4y ago
    > You can't get banned from email. If you're on Google or spamhaus's blacklist, you're effectively banned.
    • fooqux 4y ago
      You != your email address. Yes, a specific server or address can be "banned". But there is nothing stopping you from just opening another gmail / yahoo / whatevermail account.
      • bvrmn 4y ago
        And how to change a dozens accounts on others services to a new email without old one?
        • 4y ago
• edent 4y ago
  Twitter didn't scale. Not at first. Those of us on it during the early days were well acquainted with the Fail Wail. But, with a lot of engineering work - and constant upkeep - it began to handle the load. Let it be so with Mastodon. Nearly every point the author makes is subjective. That's fine, of course. But it doesn't help demonstrate that users do (or don't) want centralization. Personally, I'd rather be moderated by my local community than by a faceless American company trying to appeal to advertisers. And if I don't like their stance I can go elsewhere.
  • hedgehog 4y ago
    The scaling requirements are also unevenly distributed. Servers hosting celebrities are going to have very different load vs servers hosting 99% of people who are mostly interacting with friends/family/interest groups. The overall resource footprint with Mastodon will surely be much bigger than a fully centralized service like Twitter that can continuously identify and engineer out inefficiencies across their whole user base, not sure this is a high ranking consideration though.
    • vidarh 4y ago
      It's also not hard to solve. I've run large scale mail distribution systems, and I've run a webmail platform and been on the receiving end of large mailing list blasts. The solution in both cases is just basic divide and conquer. Mastodon the software may not be able to scale effortlessly to the largest accounts, but that's a market opportunity. ActivityPub certainly can accommodate it. If inbound volume is a problem, shard inbound activities, and zipper merge on lookup, likely maintaining a cache of the most recent n amount of entries, given this would mostly apply to things like the Federated feed for a large instance where "nobody" looks far back. If outbound volume from a single account is a problem, you similarly just split the follower list into buckets and hand off delivery to a distributed set of workers for delivery; this scales just fine with the caveat that you depend on the receiving instances being able to handle the deluge. But ActivityPub allows for batched deliveries to shared inboxes [1] you can post to, to reduce the deliveries to one message per shared inbox (which would typically best case be one per instance; e.g. the "sharedInbox" property for my mastodon account is "https://mastodon.social/inbox https://mastodon.social/inbox). Assuming all current instances has a sharedinbox property (which is likely given most of them are Mastodon, and Mastodon does), currently delivering to all the ~6m Mastodon accounts requires delivering to ~1550 shared inboxes; delivering directly to 6m wouldn't have been a problem, but it's not necessary. If handling the outbound volume from a single account arriving at your instance for delivery to a large proportion of your users is a problem, ActivityPub already has a built in solution: To find out where to deliver, you need to use WebFinger to obtain the profile urls, and get the profile urls to find the endpoints to talk to, including the inbox, which means foo@mastodon.social and bar@mastodon.social can have inboxes on different servers without even needing to do any more advanced sharding. But of course you can do the latter too and have the same inbox url mapped by your load balancer to any of a number of internal shards. This is all stuff we had to solve literally decades ago for e-mail. [1] https://www.w3.org/TR/activitypub/#shared-inbox-delivery https://www.w3.org/TR/activitypub/#shared-inbox-delivery
• thal3s 4y ago
  This is mostly an opinion piece that's critiquing issues we've only begun to solve. Also, Elon has now shown us all the absolute danger of centralized platforms. Email and web servers are a federated system and function just fine, so I'm disinclined to believe so hand-waving about how this "wont ever work."
  • krferriter 4y ago
    > Elon has now shown us all the absolute danger of centralized platforms Elon showed us this? Haven't people been talking about centralized vs decentralized systems for decades?
• encryptluks2 4y ago
  This author seems to not back up any of their claims. Like most technology you have to scale the services. If you want to look at complex examples look at how GitLab scales.
  • dewey 4y ago
    > Like most technology you have to scale the services. The blog post isn't really about how to technically scale a service though.
• kup0 4y ago
  I enjoy that most instances are small and will remain that way. There are a few that may grow to the point where scaling could become difficult, but there are ways to combat that. What the author sees as problems with mastodon (decentralization, moderation differences between instances, etc) to me are the strengths of mastodon. It's the whole dang point. Most mastodon instances aren't trying to be another twitter. The ones that are and do want to scale will inevitably run into scaling issues that any large platform full of user-generated data will run into. I'm just glad that there are different instances so when one big instance runs into scaling problems, other instances can just keep rolling along unaffected by it I think talking about mastodon as a singular entity is fraught with problems because it's simply not that
• travisgriggs 4y ago
  Isn’t it sort of an oxymoron when we try to distribute or decentralize a uniform or consistent platform? Don’t get me wrong, I’m a huge fan of decentralization. I do systems engineering. Decentralizations means redundancy. Redundancy is good. But if all you’re doing is federating a mono-platform/algorithm, then the single point of failures move to the platform itself. Decentralizing a uniform platform is like replicating the same human being lots to improve the redundancy of the human race. Ask the gene pool how well that works. Variety is good. It’s what brings beauty to the human experience.
  • ivarv 4y ago
    fwiw - there are at least two production ready servers that coexist as twitter-like social hubs - https://git.pleroma.social/pleroma/pleroma https://git.pleroma.social/pleroma/pleroma and https://github.com/mastodon/mastodon https://github.com/mastodon/mastodon. Additionally, other platforms that support the w3c's ActivityPub protocol can also interact with Mastodon.
  • lokedhs 4y ago
    The Fediverse feeds are not uniform. You may have a "federated feed" in your Mastodon instance, but the feed you see on your instance is vastly different from another instance. The federated feed is nothing more than a combined list of all public feeds from all the users on the instance. There is no way to get a central firehose, since there is a large number (I was going to say majority, but that's probably unlikely) of posts that never even leave the local instance.
  • Zak 4y ago
    There's a lot of software that speaks ActivityPub. Mastodon is the most popular, but Pleroma, Pixelfed, Hubzilla, Friendica, and several other less popular/mature options exist. Some of these, such as Friendica, are older than either ActivityPub or Mastodon, but integrated support for that protocol later.
• Zaskoda 4y ago
  This is why Bitcoin is money. Bitcoin creates an incentive to participate in running the network. Mastodon does not have such an incentive. Mastodon server admins have to either pay out of pocket or come up with a scheme to charge users. Email is similar and now most people get their e-mail from one of a few providers who exploit their data for advertising purposes. This is exactly the problem we wish to distance ourselves from in regard to centralized social services such as Facebook, Twitter, Youtube, etc. If you're not the customer, you become the product. Mastodon and the Fediverse made decentralized software that people want to use - but they didn't come up with a way to make it sustainable. Web3 has a model of financial sustainability, but they've yet to come up with any other decentralized software that lots of people want to use. These two communities focused on decentralization should really collaborate more.
  • xiphias2 4y ago
    Lightning network already works well for me, I use it on the internet regularly, and I bought a coffee with it on the street a few days ago. I would happily pay a bit for Mastodon if I needed, but for me Twitter is good enough though for now.
  • shkkmo 4y ago
    I fail to see how bitcoin or another cryptocurrency makes funding good moderation easier. "Running the network" means very different things in those two contexts. I would argue that in fact, cryptocurrencies famously face similar challenges to the fediverse with the moderation a decentralized community. Fraud, abuse and grift are rampant in the cryptocurrency community and even the good actors in that space seem to struggle to find any way to reduce that. The only limited successes there are from the centralized exchanges.
  • joecot 4y ago
    I know it's hard to imagine under Capitalism, but lots of things work without a financial incentive. The early-ish internet was powered by web forums. There are thousands of community run game servers. Wikipedia editors and contributors are unpaid. Volunteers do things because they want to see them happen and they like the community. Bitcoin has brought out the worst in many people, to the point where coal power plants were coming back online in order to handle the power demand of generating bitcoin, and miners were grabbing up as many video cards as they could, boxing out gamers from reasonable build budgets for years. Bitcoin is how society runs when it only cares about financial incentives. There are plenty of parts of society that run without a financial incentive at all just fine, and I'm hopeful the future of the web is a lot more like Mastodon than it is like web3.
• tiffanyh 4y ago
  For those not aware, the author is the creator of Flask.
  • masterof0 4y ago
    I like him, even bought his first Flask book. I just don't agree with him on this one. He (I hope is he, correct me OP) tries to make the short term scaling issues of one Fediverse project (Mastodon) the reason of why decentralization is bad or not feasible.
    • the_mitsuhiko 4y ago
      Author here: just to clarify I did not write a book on Flask, I did create the framework though.
      • masterof0 4y ago
        Oh crap, my bad, I thought Miguel did, thanks for Flask anyway.
• Aeolun 4y ago
  This guy just has issues with all the issues that directly stem from decentralization. You cannot decentralize, allow everyone to host their own server, or to join a server someone else hosts without running smack dab into the same problems. I’d argue that some of the things he sees as problems are actually features.
• Dowwie 4y ago
  > An Open Source implementation of Twitter that is significantly cheaper to run than a Mastodon host that can scale to larger user numbers should be possible. And that being Open Source would potentially permit us to see this work out in practice by letting different communities exist side by side if we can't agree on common rules Maybe Lemmy [1] with a Twitter UI is worth exploring? [1] https://github.com/LemmyNet/lemmy https://github.com/LemmyNet/lemmy
  • mdaniel 4y ago
    Heh, I'd get a kick out of seeing "twitter with downvotes" but one would have to admit that those two sites have a vastly different mental model and thus I wouldn't expect just a reskin to get it done
    • Dowwie 4y ago
      Lemmy is an engine. The Reddit UI was their first UI, and the PhpBB UI their second. Seems like a Twitter clone UI is completely within the reach of the engine, too.
• vidarh 4y ago
  I don't see anything here that is backed up with anything other than more dubious opinions. When it comes to scaling in particular, we have a counter-example in e-mail which shows scaling an inbox model + reflection (mailing-lists) to a follower-list can scale to a vastly larger audience. Of course it can, given that it decomposes neatly into a trivially parallelisable set of components of a well understood nature. It was when Twitter was still getting their fail whales, sometime around 2007, I first pointed out that parallel (might even have been on HN). There are some tricks to avoiding lag in presentation for accounts with huge numbers of followers, but there decentralisation already serves to turn a flat list of followers into a tree of instances -> subsets of followers per instance, and if/when any instances gets to a size when scaling a single instance becomes problematic, then decomposing into a virtual federated set of instances presenting a unified namespace solves that just fine (have built a mail setup doing just that - mapping <user>@<domain> to <user>@<backend shard> is not hard) The issue of "unpaid Labour" and peoples lack of desire to self host also depends on an assumption that Mastodon is inherently tied to unpaid volunteers or self hosting, but you can already pay for Mastodon hosting, and more options are sure to appear if it keeps growing. And the largest Mastodon instance is run by a non-profit getting paid by sponsorships already. And this goes straight to the issue of what happens when someone popular joins an instances and/or trust: No instance needs to accept someone who brings undue load without expecting donations or charging or just turning them away, but the converse is that trust is layered in Mastodon: You can trust someone on the basis of a history of behaviour and integrity of the node you're on, but people who need more trust can also run their own instance on their trusted domain. E.g. if POTUS wants a trusted Mastodon account, the logical choice would be to have webfinger for @potus@whitehouse.gov or similar point to a trusted instance run by the government. Nobody else can set up addresses on whitehouse.gov because it requires controlling the ability to run a webfinger setup responding at https://whitehouse.gov/.well-known/webfinger?resource=acct%3Apotus@whitehouse.gov https://whitehouse.gov/.well-known/webfinger?resource=acct%3... Conversely, if people need someone trusted to run their instance, there are any number of people who'll happily take their money to provide guarantees.
• bullen 4y ago
  The real problem is IP/DNS. Until we decentralize those we're going to have problems: http://radiomesh.org http://radiomesh.org
• TheDesolate0 4y ago
• masterof0 4y ago
  The author claims decentralization is a questionable goal, but fail at explaining why. Claims we need something in the middle, like what? He tries to link decentralization to cryptocurrencies and defis, etc... very sneaky, but lazy argument. Decentralization means freedom, self determination and power, power to host your content on your own terms, or in a place that is friendly to you, and exchange ideas with people in other communities, where they also own their data. You can have people on @socialist.social talking with @billionaries.social , but one party can't make the other disappear just because. His other argument is against Mastodon technical design, I agree , I wouldn't pick the stack the Mastodon team chose, that said, there are many activitypub enabled projects, GoToSocial, Misskey, etc... which are more resource efficient, so the failure of one project , is just a learning opportunity for another newer projects. Regarding "unpaid labor and opsec", people can rent a server from a provider that guarantees updates and uptime, or host their own, in the same way is done across the industry. I'm not claiming this is what the author is saying, but most of the criticism I see of the Fediverse comes in the flavor of "Look all those people I don't like and can't get banned, how dare them to share their ideas, let me start complaining to make them disappear".
• topspin 4y ago
  I keep having this thought that Twitter could be solved by Cloudflare or similar cloud systems that can expose HTTP APIs and cheaply serve content through global caches. Start with the principle that you own (and pay for, if necessary) the cloud resources for your piece of the social network. A standardized API that does the things you expect of a microblog; post stuff, collect replies, followers, etc. runs in your account. You control it: access, moderation (possibly delegated as you prefer), advertising, throttling, whatever. Assuming everyone is using a standardized protocol (DIDs for identity for instance) users could interact seamlessly. How hard could it be to clone Twitter on Cloudflare? Would it take more than a few thousand lines of Javascript/WASM to replicate the basic microblogging functions? A good onboarding system to automate the setup to be at least as easy(...?) as typical social networks would be necessary. Search would solve itself if the network was sufficiently popular.
  • Nemo_bis 4y ago
    In the fediverse, each instance is its own cache.
• jmbwell 4y ago
  Not sure this article is internally coherent. It starts out saying that centralization or decentralization should be an implementation detail. At the end, it advocates for a market-oriented centralization approach. Throughout, it calls out problems that seem to be features. The analogy to decentralized package hosting seems misaligned. If a package repo disappears, it breaks a tree of dependencies. If a social media instance disappears, the followers (the "dependencies") can adapt without being disrupted. This seems like a feature. On the issue of "not agreeing what it should look like," this also seems like a feature. Communities can use the platform to self-organize according to their own visions and priorities. Surely, someone who has spent time exploring the various instances available can quickly identify this. Not agreeing what it should look like is what makes it what it is. Not having to force everyone to agree what it is, is what it is. Again, it's a feature, and something fediverse users have happily "learned to live with," and moreover, embraced. Moderation and illegal content are a legitimate challenge on any platform. With the decentralized fediverse, everyone handles it according to their own policies. This does mean some admins allow it perhaps more than they should, but it also means they face the consequences directly. Admins who work to keep it out do face a daunting task in many cases, but there are tools for it. And a process of sort of natural selection already seems to favor those who have the resources and energy and motivation and skill to do it well. Seems, again, like a feature. "Mastodon is old." Not sure how this has anything at all to do with centralization or decentralization. We all use things that are old. Many old things work great. As far as others not getting their changes merged back, well, that's the benefit of forking and open source. Anyone who wants changes that aren't part of the upstream can fork it and make those changes. If enough people like them, people will move to the fork. It happens all the time. Feature. Technical challenges described are demonstrably surmountable. Market-based approach: well, we're living it, and Mastodon in particular is succeeding in the market. There are competitors like Pleroma and microblog.pub. If enough people decide Mastodon proper has "fucked up too much," then there are others right behind it ready to "step in and replace it." I dunno. There are arguable advantages to centralization, but I'm not sure this article touches on them, or at least not convincingly. Rather, it seems like the main complaints, if they're not in fact addressed by the design and operation of the fediverse, are design choices that give the fediverse advantages in other aspects. In any case, I don't see how the claim of the headline is well-supported by the points raised in the article.
• fleddr 4y ago
  I agree with the author. Mastodon is quicksand. Instances are not guaranteed to keep existing, they depend entirely on a citizen running it, scaling it, and paying for all that, often with the help of donations. Even a relatively small influx of new users may pose an existential financial threat for the instance, or a lock so that nobody new can join. Even when things are running "normally", instance owners may simply quit. Further, did you know that almost every instance regularly purges all media attached to toots? Your instance, content within it, account you created within it, the media attached to your toots, are all incredibly fragile and can disappear at any time. The main Mastodon scaling approach: creating lots of small instances, makes this problem worse, not better. You're just spreading fragility. You don't have this insecurity at Twitter, not at this fundamental level. Twitter pretty much auto-scales and your content is not lost. Sure, people may get banned, but a normy user would normally not face these existential issues. The other thing that scales poorly is moderation. Twitter is sometimes perceived as being inconsistent in its rules (or biased), but this is a 100 times worse on Mastodon. Every instance has their own arbitrary and ever-changing rules. The same applies to federation. Instance mods regularly block federation with other instances based on arbitrary decisions. Technically, the UX is inconsistent. One instance may work well whilst the other has page loads into the 10 seconds, or federated content delayed by hours. Doesn't scale, fragile, and deeply inconsistent. It has a place and I still consider Mastodon an accomplishment. But it should not be compared with any of the centralized services.
  • tpxl 4y ago
    > The other thing that scales poorly is moderation Moderation does scale poorly, which is why twitter is a shitshow and smaller forums (like this one) have a chance of being well moderated.
    • fleddr 4y ago
      I get what you're saying, but you're missing some important aspects. I wasn't commenting on the quality of moderation, instead on the scalability. Instance owners are easily overwhelmed and this problem gets worse as things grow. Second, if your point is to create a tightly-moderated instance bubble, then Mastodon's federation features will largely go unused. If that is the outcome, then there's no point to Mastodon. You can recreate such spaces everywhere, for free.
• pornel 4y ago
  I think it's necessary to separate technical problems from social ones. If necessary, the protocol can be changed to scale better (Mastodon is trying to be nice to respect a pre-existing standard, but if that standard crumbles, it's fair to come up with a better one, and IMHO it is solvable). Mastodon from the start wanted to tackle the social part of the problem about the fundamental disagreements about moderation. There's a wide spectrum between people who are keen to start Internet fights, people willing to tolerate that in the name of "free speech" principle, and people who just want to have a nice place where they can hang out without randos demanding they "debate" their right to exist. These groups will never agree what level of moderation is appropriate. Even within Mastodon communities it's controversial what the rules for federation, bans, and content warnings should be. Twitter, Facebook and others begrudgingly ended up being arbiters in political culture wars and must en masse decide what is acceptable to say. This ends up being an absurd situation where they try to balance the amount of abuse to an advertiser-acceptable level. Mastodon's answer is that you can moderate your community however you want, and cut off whole parts of the network you don't like. This is ridiculed as "bubbles", but if you don't agree — make your own anything-goes instance! It doesn't work quite well. There's still a lot to work out, but I think it's a better starting point than pleading with a billion-dollar enterprise to police the content in the way you think is right.
  • somebodythere 4y ago
    If you post on your own anything-goes instance, you will probably be defederated from your friends who are posting on normie instances. You don't have a choice but to go to a normie instance yourself and adjust your content and following to suit the instance's rules.
    • tpxl 4y ago
      If you wish to participate in society, you have to abide by the rules of the society. This is a feature, not a bug.
      • tick_tock_tick 4y ago
        But it's not society setting the rules it's a small subset of earlier admins who created some of the first popular servers. They are in no way representative of society.
• icehawk 4y ago
  Some of these arguments start on really odd premises: > Imagine you're a rather small server and suddenly Eli Lilly and Company joins your instance. Today they have around 140K followers on Twitter and they are a publicly traded company. First of all with an account that large, every one of their posts will cause a lot of load on your infrastructure. Secondly though, they are a very interesting target to attack. Why would Eli Lilly join some small instance? Why wouldn't they make their own instance? They have their own email and website after all. why be lilly@someoneelse.social when they could just be their own thing?
  • pornel 4y ago
    If they set up @social@lilly.com this solves the blue checkmark dilemmas too. It verifies their identity without implying status/endorsement/$8 flex.
    • fragmede 4y ago
      Sorta? There's nothing stopping a spammer from registering and running their own instance+user of @social@elilillypad.com, or any other variation that brandsight hasn't thought to register.
      • pornel 4y ago
        Yeah, domains as a UI element have flaws, but for better or worse they are the technological foundation of identity on the Internet.
  • this_user 4y ago
    > Why wouldn't they make their own instance? Because it's not worth the effort. If anything, there are going to a commercial providers that run servers for companies in a bundled way where all of their clients use their servers. But more likely, they won't bother at all with this, because it's not an important part of their business.
    • ilyt 4y ago
• senko 4y ago
  > On decentralized systems in particular I encourage you to read Moxie's take on web3 which outlines the challenges of this much better than I ever could I am sad that the crypto/defi/web3 crowd hijacked the term "decentralized" and now people who should know better equate the two. Web is decentralized. E-mail is decentralized. The internet is decentralized. > Mastodon encourages not just decentralization, but federation. [...] I will make the point that this is the root of the issue here. Ok, so we're actually talking about federation being bad, not decentralization as such. > I used to host a pastebin for a few years. It was Open Source and with that others also hosted it. I had to shut it down after it became (by a small percentage of users) used to host illegal content. [...] I really hard a hard few weeks when I first discovered what my software ended up being used for. We take that risk every time we open source something or run an internet service. I would not be surprised if some criminals somewhere used Flask for nefarious purposes. That doesn't mean Flask and his other software is not, on net, a huge benfit for society. This issue is orthogonal with (de)centralization, tho. > Imagine you're a rather small server and suddenly Eli Lilly and Company joins your instance. Let me rephrase that to show what a straw man this is: Imagine you're a rather small email server and suddenly Eli Lilly and Company starts hosting their mail on your instance The chance of that happening is precisely zero. WTH would they do that (and why would you let'em)? The same argument goes for Mastodon. Turning to the title of the post, Scaling Mastodon is Impossible, I agree with the author (to the extent that I know about Mastodon, anyways). Maybe Mastodon isn't up to the task, maybe ActivityPub as a protocol is inadequate, but those are technical challenges to overcome. I don't believe this dooms the entire "decentralized federated network" concept. > Wikipedia for all it's faults shows quite well that a centralized thing can exist with the right model behind it. [...] A “Not Twitter Foundation” that runs an installation of an Open Source implementation of a scalable micro blogging platform This is an interesting proposal. I worry that the cost structure - in terms of hard work, not server capacity - is way different. For all its edit wars, the basic principle on Wikipedia that someone (or a group) authors an article and thousands or millions people read it. Once the article is written, it is (for the most part) static. "NotTwitter" is the opposite - you have constant stream of new content that needs to be policed (if you don't want your town square to descend into madness). Reddit's army of moderators shows how difficult that task can be. [Meta: was the article flagged? It's got a fair number of points but is way down in the list of articles]
• goranmoomin 4y ago
  Quite interesting that a lot of comments mentioning the technical side of scaling, when the bulk of the post was about the more social side of things. While I don’t agree that decentralization itself is a dubious goal, the argument that moderation, the legal side of things, etc… is not scalable (moderation especially in its current form where the entire fediverse can report anything) seems quite true? We can’t rely on generous people hosting instances, taking legal risks and spending time for it if the fediverse wants to grow bigger than what it currently is.
• the_third_wave 4y ago
  Mastodon might not scale that well due to it being overly heavy but Activitypub scales just fine unless by scaling you mean something which makes it possible to exert control over the whole constellation by some centralised authority. I'm running a host of Activitypub services - 3 Peertube instances, 1 Pixelfed instance, 1 Pleroma instance - on a single server without any problems, mostly for experimenting with the things but the Peertube instances are used regularly. Decentralisation is not, as this piece states, 'a Questionable Goal'. It is the essence of a thriving internet, it is what makes the difference between a world-wide inTRAnet and an inTERnet. It makes it possible to pick and choose your own services or host them yourself while still being able to interoperate - the 'inter' in internet - with others. It may not be a good fit for someone's business plan but to that I can but sing a song while playing the world's smallest violin: Nae kings! Nae quins! Nae lairds! Nae masters! We willna' be fooled again!
• mike_hearn 4y ago
  It's worth noting that centralized package hosting concentrates the risk into "too big to fail" operations. This is great for as long as they genuinely are too big to fail and you can assume that someone will always step up to save the day. But the Java ecosystem went through a case where that didn't happen: JCenter/Bintray was a popular Maven hosting site for many years until one day the operator simply announced they didn't want to run it anymore and shut it down. It was a clean, phased shutdown but ultimately enormous numbers of builds and projects did have to migrate away. Now everything is even more centralized around Maven Central, which really is (hopefully) too big to fail. The financial system has a lot of experience with dependency on centralized organizations that are too big to fail. It's trading one set of problems for another. In particular the risk is that the organization starts to "fail" but not badly enough to cause a mass collective shift away. Things just degrade and become terrible but there's never a moment that overcomes the enormous activation energy needed to migrate away. With a federated or decentralized system it's easier to bleed off from an institution or service that's started failing at its core mission.
• bovermyer 4y ago
  The first half of this article had me thinking that the author was missing the point. The second half made me think, though. Specifically, about the difficulties around back pressure and moderation. The final thoughts are on the right track. A central foundation that acts as a steward for a protocol or platform, and instances of that protocol/platform are decentralized and federated.
  • Nemo_bis 4y ago
    > acts as a steward for a protocol Done: it's called the W3C. https://www.w3.org/TR/activitypub/ https://www.w3.org/TR/activitypub/ > or platform Done: it's called Mastodon gGmbh. https://blog.joinmastodon.org/2021/08/mastodon-now-a-non-profit-organisation/ https://blog.joinmastodon.org/2021/08/mastodon-now-a-non-pro...
• pwinnski 4y ago
  > My Claim: Decentralization is a Questionable Goal As oligarchs run Twitter and Facebook, decentralization seems less like a goal, and more like a necessary condition. > Imagine you're a rather small server and suddenly Eli Lilly and Company joins your instance. Why would they ever join a small indie mastodon server? Do they send email from elililly@hotmail.com too? If they can run an email server for @Lilly.com, they can run a mastodon server for @Lilly.com
• msla 4y ago
  > On the topic of moderation the very same issue is even more absurd. Some instances want uncontrolled free speech where some instances effectively are pure shit-posting instances which are completely de-federated from the most of the fediverse as a result. Other instances really like to control their content, where some popular ones such as fosstodon ban all languages than English as a result to allow moderation. There also is no real agreement on if larger or smaller instance are going to make the problem of moderation better or worse. This is an inherent problem that centralized platforms try desperately to handwave away. However, sometimes handwaving doesn't work, and there's no lowest common denominator to fall back on which will reliably keep you out of trouble everywhere; for an example that probably won't get me flamed too badly here in this decade, is it Derry or Londonderry? Just calling it Stroke City won't work if the people on the site are sufficiently enthused about their discussion forum being Correct. It gets worse the closer you get to contested claims of genocide (do you ban the people claiming it was genocide or the people insisting it wasn't?) and discussions about who gets to qualify as human. Trying to be apolitical only works if you ban all the people from one side or the other. It gets worse once you globalize, and you run into governments that are really, really keen on enforcing Correctness on certain issues, especially if you want to expand into two countries with mutually-incompatible views of which side of a given issue is Correct. Silence is not always an option. At that point, decentralization is the only solution.
• compsciphd 4y ago
  decentralized systems are always less efficient than a comparable centralized system (any insight that can make a decentralized system more efficient will do the same in a centralized system). considering how hard it is to scale centralized such systems, it makes simple sense that it be much harder to scale a decentralized system.
• aussiesnack 4y ago
  > Decentralization promotes an utopian view of the world Dystopian, actually. It promotes (and depends on) a nihilistic view that trust cannot be built, that it can only decline over time. This may be true or false. But it's not 'utopian'.
  • fleddr 4y ago
    I'd say that trust does not scale. Every online community that gets huge, tends to get ruined over time.
    • aussiesnack 4y ago
      That's an argument on the 'true' side of the ledger for said nihilism. Personally I think it's far too early to tell, with the Internet being so new, and thus far largely dominated by a single historically very singular (polite for 'odd') culture. But justified or not, giving up on genuine trust at any level of society isn't by any definition utopian.
• jaredcwhite 4y ago
  Mastodon itself as a software product/project _may_ be "impossible" to "scale" (scale to what?), but decentralization of social networking is not only not "impossible", it's actually inevitable. We'll look back at the 2010s and wonder why anyone ever thought the discourse on the web could/should become dominated by a tiny number of corporate media platforms.
  • rainonmoon 4y ago
    Does anyone really wonder why the current state of affairs has risen? Because that seems pretty naive, given how rich and influential it's made its progenitors.
  • CharlesW 4y ago
    > …decentralization of social networking is not only not "impossible", it's actually inevitable. When has any medium gone from centralized → decentralized?
    • markstos 4y ago
      AT&T was for a time the sole provider of telephone service throughout most of the United States. In the 80s it was broken up into competing companies due to anti-trust action. https://en.wikipedia.org/wiki/Breakup_of_the_Bell_System https://en.wikipedia.org/wiki/Breakup_of_the_Bell_System The companies we now know as AT&T, Verizon and CenturyLink all came from that.
      • CharlesW 4y ago
        That's a great example! Are there any mediums where this happened organically, or does this almost always happen as a result of regulatory actions? jaredcwhite: When you say decentralization is inevitable, were you talking about as a result of government intervention?
      • TchoBeer 4y ago
        It's a good thing this has lead to an increase in competition, especially local competition, and not the entire marketing being owned by a few companies instead of one and none of them treading on each other's monopolized turf.
• Finnucane 4y ago
  Perhaps scaling is also a questionable goal. Without the imperatives of capitalism--aggregating large markets to exploit--how much need is there for these services to be so large scale? For individuals, 'network effects' are useful, but not infinitely so. There are diminishing returns.
  • fleddr 4y ago
    This "need" to scale Mastodon comes from the current narrative where it's seen as a possible Twitter alternative.
  • valeg 4y ago
    A huge centralized platform is also a godsend for bad actors. You can game it and use large network effects for nefarious purposes. Also, risks (as with Musk's 8$ badges fiasco) are scaled too.
• PaulHoule 4y ago
  If it works for me and my friends why does it have to be scalable?
• hosh 4y ago
  > All of these things have one thing in common: distrust. Some movements come from the distrust of governments or taxation, others come from the distrust of central services. How about resiliency? Participation from community members? It's not always distrust that drives decentralization. I think James C. Scott's book, Seeing Like a State is worth mentioning, since it discusses the failure of centralization in more ways than simply distrust. Here's a long-form essay summarizing the highlights: https://www.ribbonfarm.com/2010/07/26/a-big-little-idea-called-legibility/ https://www.ribbonfarm.com/2010/07/26/a-big-little-idea-call... This isn't really a new idea. In building architecture, Christopher Alexander spent a lifetime writing about this. You can see some of the highlights in his keynote speech to OOPSLA '96: http://www.patternlanguage.com/archive/ieee.html http://www.patternlanguage.com/archive/ieee.html ... His ideas enable end users (residents) to be able to change their built environment while still maintaining a cohesive architectural design. Sadly, his groundbreaking ideas on pattern languages became watered down into HOA design regulations; software engineers did not heed what he talked about in that keynote; and Human Computing Interaction design dropped end-user-customizable software (like Hypercard) in favor of designs that favored aggregators, because it is more profitable (not necessarily more resilient, or better for society)
• nullcipher 4y ago
  I can't quite understand what the author's problem is. There's a bunch of conclusions without actually explaining the problem.
  • yborg 4y ago
    His problem basically is that the perfect is always the enemy of the good. Because Mastodon does not, today, solve all of social media's problems, it's not worth further effort. It's the easy hot take and there have been dozens, maybe hundreds of variations on this post on Mastodon lately, almost all of them with the implicit conclusion that it doesn't seamlessly replace Twitter, therefore the only way forward is Twitter. The same Twitter that was a service famously in search of a purpose for many years in the early days, with many pundits opining on its uselessness.
    • tick_tock_tick 4y ago
      > Because Mastodon does not, today, solve all of social media's problems Because Mastodon does not, today, solve any of social media's problems
    • pessimizer 4y ago
      > His problem basically is that the perfect is always the enemy of the good. Because Mastodon does not, today, solve all of social media's problems, it's not worth further effort. This is an unbelievably generic straw man.
• sramsay 4y ago
  I'm having flashbacks to a number of basically interminable debates in computing : big-endian vs little-endian, thin clients vs thick, windows vs mac, framework vs no-framework, functional vs imperative, monolithic kernels vs microkernels, emacs vs vi . . . In none of these cases is the debate resolved by asking: "What problem are we trying to solve?" (edited for clarity)
  • blamarvt 4y ago
    How did you forget tabs vs spaces!
• jameshart 4y ago
  > Imagine you're a rather small server and suddenly Eli Lilly and Company joins your instance. The point of mastodon is not that big companies will join small volunteer-run instances. They can run their own mastodon servers, just like they run their own mail servers and chat servers (or, more accurately, they outsource running their own mail and chat servers to Google and slack). The right place for ActivityPub to land is that businesses and institutions will host their own (or pay a SaaS offering to do it for them, like they do with email and their CMS today), individuals will sign up to shared services (like you sign up with gmail for email, or blogger.com to blog). The problem is going to be holding the barbarians at bay. All the same stuff we have to do for email will need to be deployed: Community blocklists for misbehaving servers, IP reputation, ML spam detection… And funding it will require cash which the adtech industry will spiral in and offer to provide. ‘Federate with our servers and we’ll pay you 1c for every message you allow us to post to your users…’ Server admins will resist for a while but eventually, the walls will crack. Enjoy it while it lasts, this burst of old school volunteer-run internet. It won’t survive this eternal September. It never does. As the architect of the Matrix said: Denial is the most predictable of all human responses. But, rest assured, this will be the sixth time we have destroyed it, and we have become exceedingly efficient at it
  • user3939382 4y ago
    > the architect of the Matrix Who was, by the way, an homage to Vint Cerf, creator of TCP/IP.
    • kerblang 4y ago
      Mmmm I think this is popular urban legend
      • googlryas 4y ago
        Urban legends are definitionally false. This is more of a theory but without any verification from the people who actually created the content. I would bet $1, but not more, that he is modeled after Vint Cerf.
  • 4y ago
• doener 4y ago
  "@doener guess its an basic architectural problem... I would opt to offload content to cache servers so that the many messages interchanged by the instances only contain ID's / links to the content. In an IPv6 world one even might use multicasts to send Content to the consumers more efficient... Off course such concepts of less independent instances are a different mindset. Any fully distributed system of independent instances must end up in some "maximization" of bandwidth usage. @doener also, if you think about such cache servers (editing content will be a 'problem'), one should optimise the then smaller messages interchanged between instances and between instances->users in a way, that these fit into the least count of Ethernet-frames. In an optimal world: one! Maybe in a BitTorrent way, users could also act as "cache servers" for content." https://social.tchncs.de/@schnedan/109344407024988466 https://social.tchncs.de/@schnedan/109344407024988466
• markstos 4y ago
  Mastodon is messy. The world is messy. We have cities with different rules, different mayors, different odds of existing in 50 years. It's nice to have all the cities follow all the same rules and customs if you agree with them, and nice to have another city to move to if you don't. Email as a decentralized medium has survived for decades. You use a big provider like Gmail, choose a host in another region, or an organization like Proton Mail that does thing somewhat different. It's OK that Mastodon is messy and at times chaotic. It's organic.
• tbugrara 4y ago
  Author lacks imagination.
• whalesalad 4y ago
  I realize that the word "scaling" here is used in the human/community sense, but for some reason (considering Mastodon is written in Rails) I am reminded of this gem: https://web.archive.org/web/20080107085941/http://www.zedshaw.com/rants/rails_is_a_ghetto.html https://web.archive.org/web/20080107085941/http://www.zedsha...
  • ilyt 4y ago
    The guy have massive "I wrote shit that happened to be popular so I must be good" issue.. I still remember having to run his malady of a web server when we still had big Ruby team. I guess it might've been "good" compared to other things Ruby... and the clown car mentality like > had this stupid setup where the people deploying Rails didn’t have root access. I told Kevin that this was stupid and apparently it was Kevin’s idea because he told me “fuck you”. Like, holy shit, even at dark times of cgi-bin we didn't had much of this idiocy and he's taking issue with people not wanting to deploy stuff as root...
• varelse 4y ago
• nyx_land 4y ago
  > All of these things have one thing in common: distrust. Some movements come from the distrust of governments or taxation, others come from the distrust of central services. This isn't framing the problem the fediverse is solving accurately. What fedi solves isn't not trusting anyone, because it's not a fully P2P network. Rather, it's more about having accountability for the services you use. With a massive centralized social media platform like Twitter where it needs to effectively scale up to be able to accommodate the entire world, it's impossible for a service like this to exist without either being run by a government or a corporation. In the US, the two are more or less the same anyways since we've outsourced most of our infrastructure to the private sector and have been doing that for decades. And in the case of a private entity controlling one of these services that need to scale massively, it's effectively impossible for them to not be incentivized to do all the things that make people not trust them -- mining users' data, etc. On fedi, you know who your server admins are; they're human beings that you can message if you have a dispute with another user, or there's an issue with the service itself, or whatever. If an instance admin does something shitty, they can be held accountable for it and can have their reputation ruined on fedi, which isn't a perfect system since it leads to things like people being defederated unjustly, but I would say it's far more scalable socially to have a patchwork of different small to medium sized servers run by individuals or teams of people rather than faceless corporations that largely automate all of their user-facing interactions (algorithmic bans and support) where it's basically impossible to hold them accountable for anything unless you have millions of dollars to burn in court. It is true that centralization/decentralization are implementation details for solving a problem, but this is a case of technological solutions not being sufficient for cultural problems and vice versa. Fedi, for all the problems I have with it, is however closer to striking a balance between solving problems on both of these fronts because it returns the internet to being a community-driven network that has its own culture, and has more ability to handle bad actors or failures in the network with federation as opposed to earlier eras in the internet where for example everyone was using their own separate forums and the centralization of the internet onto social media platforms was really an inevitable consequence of that structure of the internet.
• gigatexal 4y ago
  Well this is alarmist. I haven’t read the ActivityPub spec or code but the main failing in the current implementation of Mastodon is the codebase is not conducive to horizontal scaling. There will probably be work in that domain sooner or later. Or it’ll never really take off. Either way I like how it’s small and niche. Let the normals stay on Twitter and they can keep the advertisers, too. Moderation is done by users of a node or a server admin or admins. And from what I can see admins block on a node level and users can block folks at the user or a whole subdomain/node.
• Animats 4y ago
  There are several issues here: finding, identity, storage, and delivery. They're somewhat separable. Finding is "where do I go to get X". We have URLs, which assume a specific server. There are content-based keys, such as DOIs and URIs and hashes. But how do you find where the info is stored? Google? Something like DNS? Something else? This is the hardest problem. What Youtube really sells is "discovery" not streaming hosting. There are lots of streaming services, but you won't get the views. Identity has all the usual problems. If people can create lots of identities at low cost, there will be spam and worse. No good answers there. China has this fixed, but they don't do anonymity. You need a government ID to connect to anything. Storage is the big cost problem. Where does all this stuff go, and who pays for it? IPFS was supposed to be the distributed answer to this, and Filecoin was supposed to be the way to pay for it. That didn't work out. On the other hand, if someone wants it out there, then maybe they have to pay to store one copy. Delivery can be distributed, but do you want to? Bittorrent was the prototype. Peertube is another peer to peer way to do it. Each video has a home server, and large numbers of people watching the same thing won't overload it because anyone watching the video also serves it. It works OK but is not as smooth an experience as YouTube. Plus it runs down your battery and runs up your bandwidth usage. Bandwidth is much cheaper in data center bulk than out at the end of a cell connection. Maybe do something like that but with ISP level caching servers, all serving each other. Sort of like Cloudflare / Akamai.
  • klabb3 4y ago
    Thank you, this is constructive. Let me complement with some more details: Privacy. Messages in a private chat or group should not be visible to the operator or others. This likely necessitates e2ee, but it's possible that some multi-party protocol could protect from malicious operators without going full on e2ee (difficult on web). Integrity: you don't want rogue operators to be able to falsify content. While you can't stop any website from faking content, you can stop it from propagating by having signatures and verifying them. Availability: you should be able to post and fetch at all times, which necessitates aggressive caching. Signatures (from above) are useful here, because you can use untrusted storage layers. Security: how do you prevent an instance operator from stealing your account or posting on your behalf. Abuse: no system is anywhere near perfect, but the ones that come close are extremely centralized and use fingerprinting and behavior analysis. We probably don't want that, so what else? Note that all of these are deeply tied into the identity problem. We would really need a form of PKI for real people (but without exposing real world identity). If people actually control their private keys, all of the above is greatly simplified. But even in defi, where custody of your private keys have a HUGE direct monetary value, most people don't want/know/care to set it up. So we have huge challenges.
• javier_e06 4y ago
  The post is a collection of rambling thoughts. The author should take a look at Linux Torvalds talk with Google (2009). https://www.youtube.com/watch?v=4XpnKHJAok8 https://www.youtube.com/watch?v=4XpnKHJAok8 back and day when development was centralized (no cloning). The concept of "Trusted Circles" is at the core any resilient information exchange and it is by definition: Decentralized.
• ilyt 4y ago
  Let's start from the fact that "decentralization" in "moderator of server you've chosen to dwell decides what you can see and what you can not" is terrible idea from the get go. Whether decentralization for social twitter-like sites might work is up to discussion, but Mastodon is just bad try at that. You're trading consolidated moderation by one rule for thousand little fiefdoms, each with different rules and waging ban war on eachother. Empower the users to filter and pick what they want to watch, not moderators
  • rexpop 4y ago
    > moderator of server you've chosen to dwell decides what you can see and what you can not That's already the case under the "moderation by one rule" paradigm.
  • hedora 4y ago
    > Empower the users to filter and pick what they want to watch, not moderators I can imagine a friends and family instance that solves this: only invite people I wouldn't need to moderate. Honestly, though 1 person per instance makes more sense to me. There needs to be a way to drive the (time and money) cost of hosting an instance to zero. There probably also needs to be some sort of web of trust style reputation system. (Or just a way to whitelist users instead of blocking them.)
    • ilyt 4y ago
      The G+ "circles" idea made sense. Frankly distributed social medai should be more akin to Google Reader and less of what Mastodon tried to be. Like, have server have its users stream, still allow moderators to block incoming comments on that (aside from mod ego trips bad actors will always exist), but allow user to subscribe whatever they want without mod touching that path. So say mod would be allowed to block @server or user@server and that would cut that from public stream but not from people that subscribed to the guy or to feed from that server's public streams. That way you can as mod still choose to not have politics poison your stream of dunno, amateur woodworking twoops, but it doesn't infringe on your user's freedom to see and talk with whoever they want. > There probably also needs to be some sort of web of trust style reputation system. (Or just a way to whitelist users instead of blocking them.) Frankly blockchain here would be the sole case where that technology might've had any benefit, even if blockchain was just used to store user's identity data and facilitate profile transfer. Maybe separate one if some servers want to make a blocklist in "if at least X out of Y servers see user misbehave, permaban" way. But eh, what stops someone from making 20 accounts on 20 different federated servers and report same post ? Distributed reputation is hard. "just" downvote/upvote karma system is frankly silly to be anything else than brag number, as essentially you just need to make popular post to counter few unpopular ones. I think there should be a system to attach up/downvote count to a post, that makes it easy to cut on low effort/dumb content without much mod input, but making that not be gamed cross-server would be hard.... and per-server score is less and less useful the smaller server is/
• bawolff 4y ago
  > Decentralization promotes an utopian view of the world that I belief fails to address actual real problems in practice. Its interesting - most recent examples this is true, but it seemed to work well in the past. Bit torrent is a staggering success. Email is showing some flaws now a days, but how many other application layer protocols do you know that were designed 40 years and still widely in use. Although controversial, i would actually say original bitcoin was also a success in the sense it did what it intended to do beyond anyone's wildest dreams (whether or not that is a good thing is a different conversation) Decentralized protocols seem to have a long history of success. Somewhere that seems to have flipped.
  • ilyt 4y ago
    > Bit torrent is a staggering success. There is no way to inject spam and ads to the torrent and little incentive to fake seeders/peers to make people download the wrong one. That's why. It's not a communication method, it uses other communication methods to distribute torrents/magnet links > Email is showing some flaws now a days, but how many other application layer protocols do you know that were designed 40 years and still widely in use. Frankly we nearly had that in blogosphere. You could federate what you want to watch via RSS. You could discuss under the post. There was even mechanism to get which post referred yours via pingbacks (but of course got removed due to of course being way to distribute spam. The thing it lacked is some kind of place to be endlessly fed whatever is now popular (as it is trend for every popular social platform) and general seamless usability. But it put entirety of curation up to the user, don't like the blog ? Don't subscribe it, no mod to tell you what to do nor someone injecting shit into your stream because mods decided this thing need to be promoted now
• deleted 4y ago
  [deleted]
• deleted 4y ago
  [deleted]
• Barrin92 4y ago
  The article makes a correct distinction between decentralization at a technical and social level that is direly needed. We ought to make a much stronger distinction between distributed computing and decentralized communities. It's more or less a quadrant. Torrent systems and Mastodon are both decentralized and distributed. Twitter is both technically and socially centralized. Reddit is technically centralized but socially federated, and contrary to popular intuition a lot of cryptocurrencies are centralized, global state machines on distributed hardware, not actually decentralized socially which would defeat their purpose for existing. What people are after in social networks increasingly is local communities. Whether it runs on protocols or platforms doesn't really matter. I'd agree though with the authors and Moxie's take that protocols have a lot of problems due to their slowness and technical hurdles. In reality they also tend to coalesce into platforms anyway due to economic pressures. The most appealing quadrant for social networks I think is the Reddit one. Some platform with a baseline and minimum of largely legal rules but local moderation. Which is essentially how real world federations work anyway. If it's a new or existing one I don't know but I could well imagine that Twitter or Facebook more and more devolve power towards regional communities and create better and fine-tuned moderation tools. Telegram and Discord are already examples of this as well.
• talideon 4y ago
  This falls apart with any comparison with email. Email is a pure push protocol that assumes all actors are trustworthy. Messaging in the vein of ActivityPub, is pull: you might be able to opt in to receiving messages from anyone, but you don't have to. You can subscribe to people and only get messages from them and those they boost. And you see who's boosting stuff you don't want to see and can stop following them or talk to them. That reduces the problem down to a narrower set of social problems, and social issues are hard, but in a different way: you've just discovered your friend is a jerk rather than them being an open relay.
• teawrecks 4y ago
  > Decentralization promotes an utopian view of the world that I belief fails to address actual real problems in practice Email. > a better alternative...would be...an Open Source implementation of Twitter...that being Open Source would...[let] different communities exist side by side if we can't agree on common rules. Isn't that literally Mastodon? If you don't think mastodon's tech will scale or you don't like how the protocol is developed, that's one thing. But I don't understand saying that you're anti-decentralization, and then proposing decentralization as the alternative.
• spion 4y ago
  > Even today I still receive emails This folks is all you need to read from the above article. Yes, I'm exaggerating a little bit, but we keep forgetting that federated systems have worked before and will continue to work. This is not some failed idea.
• salusinarduis 4y ago
  My issue with Mastodon is deployment. Why do they have all these silly little services doing different tasks? Inevitably someone's going to rewrite it all in Rust and it's going to handle 100x (without even being optimized/load balanced) whatever it's doing right now and it will be as simple as turning on Postgres and launching a binary. From there people can choose to run external services to further handle load, but right now just to launch a small instance, needing all the crap that comes along with the node.js world sucks.
  • pdntspa 4y ago
    But... but... everybody loves microservices! (Or so they want you to believe)
• friend_and_foe 4y ago
  > The ActivityPub is a pretty messy protocol, and it also appears to not have been written with scalability in mind much. The thing does not scale to the number of users it currently has and there is probably no trivial way to fix it up. Citation needed, the thing seems to be scaling just fine for the needs of the current userbase. > There also is no real agreement on if larger or smaller instance are going to make the problem of moderation better or worse. That's the whole point. If you don't like how a community conducts itself you're free to block them as a user, or even as an admin. > A service like Twitter is much cheaper and easier to run if it does not have to deal with federation on a technical level We had that, it was called the internet. Everyone went to Twitter, because they want to talk to everyone else without a hundred accounts. We still have that, it's called Mastodon (or pleroma...), you don't have to federate if you don't want to. You don't have to like it. If you have ideas, implement them. But don't tell the network to change itself because, as someone said once, they were there before you and they'll be there long after you leave.
• masukomi 4y ago
  I wrote a rebuttal to this: https://weblog.masukomi.org/2022/11/14/a-rebuttal-to-scaling-mastodon-is-impossible/ https://weblog.masukomi.org/2022/11/14/a-rebuttal-to-scaling... The short short version is that it scales just fine if you use it the way its intended. The protocol provably scales just fine (contrary to his claim), the real problem with scaling is that it violates the core idea that Fediverse server apps (all of them) are written with the idea of serving a community, not hosting the next multi-millon user twitter competitor. There's more to it than that, and there are a lot of straw-man arguments and contradictory claims in that article.
• SevenNation 4y ago
  > Yet there is the belief that you can somehow create a coherent experience into a “whatever”. Whatever it is actually. My first mastodon instance was de-federated by accident from my current instance. I moved to that instance though because many other hackers in the Open Source space did, and unlike Fosstodon it seems to allow non English content which I do care about quite a bit. (After all my life and household is multilingual and I don't live in an English speaking country.) Yet that instance still defederates qoto and I'm guessing because qoto permits unpopular opinions and does not block servers itself. Given the history of email, I'm surprised there hasn't been more about the economics of spam, blacklists, and so on. Substitute the email-specific parts of the following article with Mastodon-specific terms and concepts, and it doesn't paint a very pretty picture. https://blog.lopp.net/death-of-decentralized-email/ https://blog.lopp.net/death-of-decentralized-email/
• raydiatian 4y ago
  > Open Source implementation of Twitter Isn’t this exactly what Jackattack is working on right now?
• jaimex2 4y ago
  Should have just said Mastodon is the new Google+ It'll serve a niche group then be put out to pasture.
• mastazi 4y ago
  TL;DR: "I used to be on Twitter and now I'm trying to force Mastodon into being Twitter, ignoring what Mastodon actually is and why it was built, and now I'm upset because it didn't work". Mastodon doesn't need to scale like Twitter, the point of Mastodon is many instances, not a single giant one. Sounds like Mastodon is not for you. Good, you can leave now. Mastodon's mission is not, and never has been, replacing Twitter.
• summerlight 4y ago
  I'm generally fine with Mastodon's approach. But from the inception of the internet, I've seen lots of community manager who took their moderation work mostly as their hobby. Then the community grew quickly for some reason, moderation became nearly unsustainable. A few of them were able to scale out their moderation framework but most of them just decided to sacrifice their personal life to the end, eventually burnt out then abandon their child. I wonder if the Mastodon community found a structural solution to handle this kind of issue.
  • Nemo_bis 4y ago
    See: How to make the fediverse your own https://cloud.medlab.host/s/JKfLGa5zxPqoT9r https://cloud.medlab.host/s/JKfLGa5zxPqoT9r (by social.coop folks).
• est 4y ago
  > Or that the most controversial and replied to issue is about optionally disabling replies to posts like on Twitter. This is perhaps the greatest weakness of mastodon. On one hand, there need to be some mechanism to disable harrassment, on the otherhand, it's practically impossible on "decentralized" networks.
• eduction 4y ago
  The writing here is extremely poor. Author should delete and rewrite as a six paragraph post. (What was that segue about python package management intended to show?)
• alwillis 4y ago
  Mastodon will scale if it has to. Mastodon and ActivityPub have been flying under the radar for years and now this is its close-up. The ActivityPub spec hasn’t been updated since it became a W3C standard 4 years ago; no doubt it needs to be updated and probably will be, now that so many users are now on ActivityPub servers. There are going to be growing pains but that’s to be expected. People remember the days of the Fail Whale, when during a big news event, Twitter would just stop working. Timing is important—there’s enough visceral dislike for Elon Musk and users getting tired of billionaires upending things. It’s not going to happen overnight but I expect good things to continue to happen for distributed social media.
• mempko 4y ago
  Wow, this person doesn't get it. You can tell when they start talking about 'market based approach'.
• pkdpic 4y ago
  Am I the only one who read the title of this post and had the knee-jerk reaction of "good"? The articles point seems to have its flaws and maybe the main point is that mastodon will be ruined not saved from ruin by the scaling problems being outlined though so :shrug-emoji: I just wanted to note that initial reaction / feeling.
• xg15 4y ago
  > Centralization or decentralization should really be an implementation detail of the solution to an actual problem. > For that particular problem the solution might be one of those two things, or something in the middle. But out of principle it should be neither of those two things. So which one is it? Pragmatic implementation detail or "out of principle"? Also, if the author prefers a solution somewhere in the middle, wouldn't the Fediverse be the perfect example? Full decentralisation would be some kind of p2p network or possibly blockchain thing where there are no servers at all. Instead, each server on its own is perfectly centralised, it's just the community at large that is decentralised.
• e12e 4y ago
  > Decentralization promotes an utopian view of the world that I belief fails to address actual real problems in practice. People have mentioned email, but what about DNS? RIPE/ip4 routing? Small, federateted invite-only, self-hostable communities seem exactly the right fit for social media. The gatekeepers know their users (invite only). Instances can be small, easier to agree on a culture/moderation policy. Federation allows cross-pollination of ideas across communities. For example, with DNS most people (with internet access, I think) can register a domain, but not everyone can register a .no/Norwegian domain. The organization (NORID) that has delegated authority for the zone, knows Norway. It has a priv.no subdomain, and it knows about government entities an municipalities that we've decided should have special status. This is a pretty great example of efficient decentralized authority. There are other registrars that have decided to organize things differently, like Tuvalu - but that doesn't really impact NORID as such.
• Niksko 4y ago
  Two things stuck out to me as not very good arguments in this article. One seemed to be that different moderation standards make a coherent experience impossible. I think this is a fundamental difference of the fediverse. There is no one experience, and that's a feature not a bug. I think it also makes out that by banning other instances outright, instances will be largely disconnected. I think this ignores that, more or less, people agree with what is acceptable and what is not. And if they don't largely agree, maybe those groups of instances should be disconnected. I think it also assumes that this disconnected experience is somehow opaque to users, when I think it's pretty easy to figure out that you're only seeing Toots from your own server. I think that's also not a bad thing. Again, if you set aside the idea that everyone has to be on the same experience, viewing the same content, the decision calculus becomes easy. Are you happy with the depth and breadth of the content you're seeing? No? Then move instances (which is easy). My other criticism is about legality. The US has section 230 of the telecoms act, which basically allows internet companies to moderate their websites, without having to be responsible for what is posted there by users. Basically, the argument that the law will somehow sink the fediverse ignores that we made it this far. The web was allowed to flourish, in part because what you say online is largely your responsibility, not the responsibility of the sites that host the content (with a few exceptions). I'm not sure how Mastodon meaningfully changes that.
• antonyh 4y ago
  This feels a little like "the cathedral and the bazaar" in the sense that it's either controlled by a small group or allowed to fragment and grow organically in a less structured, organised-chaos model. There are pros and cons either way, but finally it seems that we have a choice. The Fediverse is here and will remain, and seems to have enough gatekeeping to limit federation of unwanted/spammy/malicious servers. Centralised forms of social media will continue to exist but this new landscape will change how they work and will change the content created and shared.
• lakomen 4y ago
  I never understood the appeal of Twitter. I hosted a Mastodon instance 2 years ago for the African disapora and 1. No one used it and 2. I had to keep it up to date and couldn't remove the Mastodon copyright bs. So I just stopped. I pay with my own resources that someone else benefits. I don't think so. Same with Lemmy. I pay for them benefiting from the cream of my content on their meta aggregation site. I don't think so.
• jamesgill 4y ago
  Perhaps I’m being naive, but—is ‘scaling’ the point, or even necessary?
• naravara 4y ago
  I think the author is coming at this with a focus on the crypto scene, where it really is deeply rooted in distrust, but I see the value of decentralization in the Mastodon/ActivityPub more as one of a desire for more variation and competition than the centralized model allows. Trust is part of the deal, and Mastodon is a bit biased towards people with a particular fixation on moderation practices and casting out bad-actors to the exclusion of a lot other things. But for old heads, I think the real goal is for something more like the blogosphere or usenet groups or forum cultures where it wasn't about lack of trust but the ecosystem felt more dynamic because there were simply a lot of people just kind of doing their own thing in different ways. It iterated better, and it iterated in ways that were more centered on the content creators and consumers instead of a platform clearinghouse.
• mrdrozdov 4y ago
  This is such a click-bait title and has nothing to do with scalability.
• dimensionc132 4y ago
• Hakashiro 4y ago
  I think that Mastodon's problems are less than 10% technical. That's not to say ActivityPub isn't a pretty trash protocol, insecure by default (no E2EE), wide open to abuse, and very wasteful (because posts have to be replicated in every server, making unnecessary copies of a post that aren't then distributed via P2P). But the social issues of pretty big Mastodon instances blocking other smaller instances with complete impunity, means the Fediverse as a community (not a protocol) encourages bullying and isolation. I find this extremely troublesome in the wake of the Capitol attack, fake news campaigns, State-sponsored disinformation campaigns and everything else that is wrong with social media today. You will get your server banned from other servers if your server doesn't ban the servers these servers ban (second-grade sanctions). You can get banned for not moderating enough, or for incompatible code of conducts. You can get banned for an instance that is too small, as other admins believe you are using it for ban evasion. The worst part is the user can't know if the posts they make will be seen in other servers. They can't know if sending messages to other user will work. They can't opt out of these admin-imposed forceful defederations. And even worse: it is absurdly simple to push illegal content like child pornography into other servers, making these servers store illegal content in their hard disk. And the only way to avoid this is to have server administrators manually see the offending post and possibly block the offending domain. Perhaps with the help of users reporting this post. If Mastodon is the best Twitter alternative, I think nothing can replace Twitter as it is today.