3 ms·
A certain degree of overhead just has to be there, a distributed system needs to exchange information between nodes after all. And both the protocol and many im
by detaro 3y ago
A certain degree of overhead just has to be there, a distributed system needs to exchange information between nodes after all. And both the protocol and many implementations leave room for optimization at least. I think tackling that is one of the long-term challenges for the ecosystem, but I wouldn't say it is categorically "not scalable".
EDIT: As an example, right now in Mastodon federation traffic is handled by sidekiq queues running Ruby. Someone makes a post -> tasks to send that information to all their followers instances get queued, which then send individual HTTP requests to all those instances. This to me feels like something that could be moved into some smaller and more-efficiently-written component that batches requests, makes sure to reuse connections, ... at least for paths that see a lot of traffic.