4 ms·

I would love to see some data about what you speak of, but a raw ActivityPub activity is maybe 1K in size. Propagating 1K sized payloads from 10K people (let's

I would love to see some data about what you speak of, but a raw ActivityPub activity is maybe 1K in size. Propagating 1K sized payloads from 10K people (let's say that gives us about 10 per second) should not be that big of a deal in the age of streaming. I think unofficial censuses would put the total instance number at around 25000, so that's about the maximum number of outgoing messages a server would send. 25000(instances) * 10(messages/second) * 1000(bytes) = 256000000 bytes/s = 256MB/s of outgoing traffic Where Mastodon really gets it wrong is with the incessant caching of media that every instance does, and which is what maybe you are inadvertently thinking of.

armchairhacker 4y ago
It may just be Mastodon which has the problem. Still, competitors should make sure to not re-create the problem. I don't remember the specific comment but something about how one of the top Mastodon servers was transmitting 100TB of data a month, or some other else unreasonably high number. This server only had ~10,000 users.
- mariusor 4y ago
  If my math doesn't fail me that's almost an order of magnitude lower than 256MB/s. :D I'm suprised I got so close with my back of the napkin calculation. But what would you consider reasonable data usage for 10K users? Is it maybe possible that a technology that's trying to demonopolyse social networks is probably not optimised for high user counts on individual servers?
  - armchairhacker 4y ago
    I don't know how much data Hacker News transmits but I'm certain it's under 100TB/month. I do know HN has >6 million requests per day and only runs on 1 server (https://news.ycombinator.com/item?id=28478379 https://news.ycombinator.com/item?id=28478379). I'm sure Mastodon requires more data-per-user than HN, and a server with 1M+ users will be expensive. But I don't think 10,000 users is that much, and a server with that many shouldn't cost $1000+/mo to operate.
    - charrondev 4y ago
      A couple things that I would imagine add to the cost with federation: Bandwidth costs for user content. You aren’t just hosting the content of 10k users but caching and serving the content of everyone you federate with. Either that or the software would have to be very inefficient.
      - mariusor 4y ago
        > caching and serving the content of everyone you federate with This is a Mastodon quirk. Eugen made the decision that everything must be locally cached for the reason that big instances should not have a "slashdot effect" on remote, smaller instances. In my opinion that's wrong and his solution is actively working against the idea of "federated network", as the fediverse wants to be, and as the internet already is. The user facing problems stemming from this decision is (from the top of my head) the fact that you can't view another user's past posts if his server didn't federate with yours. That's such a low bar of improvement that it baffles me that nobody bothered to fix it so far.
    - mariusor 4y ago
      I don't know of a serious hosting provider that would have an issue with one of their customers serving 100TB volume. The problem with the monthly costs is probably due to requiring quite beefy hardware to run Mastodon's inefficient rails/sidekiq application.

armchairhacker 4y ago

It may just be Mastodon which has the problem. Still, competitors should make sure to not re-create the problem. I don't remember the specific comment but something about how one of the top Mastodon servers was transmitting 100TB of data a month, or some other else unreasonably high number. This server only had ~10,000 users.

mariusor 4y ago
If my math doesn't fail me that's almost an order of magnitude lower than 256MB/s. :D I'm suprised I got so close with my back of the napkin calculation. But what would you consider reasonable data usage for 10K users? Is it maybe possible that a technology that's trying to demonopolyse social networks is probably not optimised for high user counts on individual servers?
- armchairhacker 4y ago
  I don't know how much data Hacker News transmits but I'm certain it's under 100TB/month. I do know HN has >6 million requests per day and only runs on 1 server (https://news.ycombinator.com/item?id=28478379 https://news.ycombinator.com/item?id=28478379). I'm sure Mastodon requires more data-per-user than HN, and a server with 1M+ users will be expensive. But I don't think 10,000 users is that much, and a server with that many shouldn't cost $1000+/mo to operate.
  - charrondev 4y ago
    A couple things that I would imagine add to the cost with federation: Bandwidth costs for user content. You aren’t just hosting the content of 10k users but caching and serving the content of everyone you federate with. Either that or the software would have to be very inefficient.
    - mariusor 4y ago
      > caching and serving the content of everyone you federate with This is a Mastodon quirk. Eugen made the decision that everything must be locally cached for the reason that big instances should not have a "slashdot effect" on remote, smaller instances. In my opinion that's wrong and his solution is actively working against the idea of "federated network", as the fediverse wants to be, and as the internet already is. The user facing problems stemming from this decision is (from the top of my head) the fact that you can't view another user's past posts if his server didn't federate with yours. That's such a low bar of improvement that it baffles me that nobody bothered to fix it so far.
  - mariusor 4y ago
    I don't know of a serious hosting provider that would have an issue with one of their customers serving 100TB volume. The problem with the monthly costs is probably due to requiring quite beefy hardware to run Mastodon's inefficient rails/sidekiq application.