3 ms·

No. What I mean by "doesn't scale" is the idea of having the fediverse dependent on millions of small instances and has nothing to do with programming language

No. What I mean by "doesn't scale" is the idea of having the fediverse dependent on millions of small instances and has nothing to do with programming language of choice. Even the Pleroma devs have said that their system is not suitable for deploys with more than 25k users.

ndriscoll 3y ago
But you wouldn't need millions of small instances if you had better scalability per instance. Allegedly, Twitter had ~600 requests/second when it had ~350k users[0]. That's completely trivial, and as long as you use hardware from within the last 5-10 years (and, importantly, new NVMe SSDs), you should be able to handle at the very least 10x that on one server, meaning one server ought to be able to handle more like 3.5 million users at the very least. If you know what you're doing, you can do a lot more (most likely you'd end up limited by how much bandwidth you can get affordably). I don't know much about Pleroma, but it sounds like it's also very slow (i.e. it struggles with ~60 requests/second)? I've done some experiments with Phoenix and know it can do a lot more assuming it's not on a raspberry pi or something. Without knowing much about why they might have made this schema the way they did, this ticket[1] has me raising eyebrows for example. [0] http://highscalability.com/blog/2009/6/27/scaling-twitter-making-twitter-10000-percent-faster.html http://highscalability.com/blog/2009/6/27/scaling-twitter-ma... [1] https://git.pleroma.social/pleroma/pleroma/-/issues/3139 https://git.pleroma.social/pleroma/pleroma/-/issues/3139
- rglullis 3y ago
  The problem of large instances is not just on the amount of users that you have, is on the amount of followers that your users have. E.g, imagine you have one user in your server that has 100k followers, spread around ~1000 instances. Each new message from this user triggers a push to these instances, and each of these instances will then make a request to you at the same time. Then, add to this that each of followers may trigger another request to your server in case the message has an embedded card or image. On a centralized system like Twitter this does not happen, on the fediverse this may lead to a self-DDOS.
  - ndriscoll 3y ago
    If you're pushing, why would the other instance pull from you? Do one or the other. You also don't need to push for each post. You can batch push once per second or whatever. Push the image too so they don't request from you. Again, this may not be how mastodon works, but mastodon is clearly designed poorly. I have systems at work that have no problem gossiping thousands of updates per second to hundreds of subscribers. You also don't have to have a single global federated network to have a useful system. If North America were isolated from Europe or India that would still be useful to tons of people. Or people might segregate into communities based on interest, and don't need to send the same tweet to different communities since it's not relevant to both groups.
    - rglullis 3y ago
      > If you're pushing, why would the other instance pull from you? Because instances are not always assumed to be online, or you don't want to push to all the connected instances... > I have systems at work that have no problem gossiping thousands of updates per second to hundreds of subscribers. So, I agree with you [0], but it seems that the Lemmy devs think it is not necessary. Be a kind soul and show your PR? [0]: https://github.com/LemmyNet/lemmy/issues/3245#issuecomment-1601644229 https://github.com/LemmyNet/lemmy/issues/3245#issuecomment-1...

ndriscoll 3y ago

But you wouldn't need millions of small instances if you had better scalability per instance. Allegedly, Twitter had ~600 requests/second when it had ~350k users[0]. That's completely trivial, and as long as you use hardware from within the last 5-10 years (and, importantly, new NVMe SSDs), you should be able to handle at the very least 10x that on one server, meaning one server ought to be able to handle more like 3.5 million users at the very least. If you know what you're doing, you can do a lot more (most likely you'd end up limited by how much bandwidth you can get affordably). I don't know much about Pleroma, but it sounds like it's also very slow (i.e. it struggles with ~60 requests/second)? I've done some experiments with Phoenix and know it can do a lot more assuming it's not on a raspberry pi or something. Without knowing much about why they might have made this schema the way they did, this ticket[1] has me raising eyebrows for example. [0] http://highscalability.com/blog/2009/6/27/scaling-twitter-making-twitter-10000-percent-faster.html http://highscalability.com/blog/2009/6/27/scaling-twitter-ma... [1] https://git.pleroma.social/pleroma/pleroma/-/issues/3139 https://git.pleroma.social/pleroma/pleroma/-/issues/3139

rglullis 3y ago
The problem of large instances is not just on the amount of users that you have, is on the amount of followers that your users have. E.g, imagine you have one user in your server that has 100k followers, spread around ~1000 instances. Each new message from this user triggers a push to these instances, and each of these instances will then make a request to you at the same time. Then, add to this that each of followers may trigger another request to your server in case the message has an embedded card or image. On a centralized system like Twitter this does not happen, on the fediverse this may lead to a self-DDOS.
- ndriscoll 3y ago
  If you're pushing, why would the other instance pull from you? Do one or the other. You also don't need to push for each post. You can batch push once per second or whatever. Push the image too so they don't request from you. Again, this may not be how mastodon works, but mastodon is clearly designed poorly. I have systems at work that have no problem gossiping thousands of updates per second to hundreds of subscribers. You also don't have to have a single global federated network to have a useful system. If North America were isolated from Europe or India that would still be useful to tons of people. Or people might segregate into communities based on interest, and don't need to send the same tweet to different communities since it's not relevant to both groups.
  - rglullis 3y ago
    > If you're pushing, why would the other instance pull from you? Because instances are not always assumed to be online, or you don't want to push to all the connected instances... > I have systems at work that have no problem gossiping thousands of updates per second to hundreds of subscribers. So, I agree with you [0], but it seems that the Lemmy devs think it is not necessary. Be a kind soul and show your PR? [0]: https://github.com/LemmyNet/lemmy/issues/3245#issuecomment-1601644229 https://github.com/LemmyNet/lemmy/issues/3245#issuecomment-1...