3 ms·

My question is, why do you need separate processes to handle mostly async operations? You need to do HTTP requests and database queries, that is mostly waiting,

by pimeys 4y ago

My question is, why do you need separate processes to handle mostly async operations? You need to do HTTP requests and database queries, that is mostly waiting, there is no CPU needed for this. You can do it with one process and millions of tasks operating in concurrent fashion. I don't really know how Mastodon does this with the Sidekiq, but what I've seen people with a very small amount of activities (we're talking about hundreds of thousands, or a few millions), need to do manual scaling of Sidekiq processes or threads. As a user, you should not be tuning these metrics to handle this kind of activity. A single beam process could handle all the WhatsApp messaging back in the days, so it can easily handle our relatively tiny fediverse servers with much less users.

rektide 4y ago
One of the main wins I've contributed to company after company is taking monoliths & making multiple instances of them. Yes it can run one big process of everything, and does when I find it. But some routes or subservices are slow or hog cpu or use way more memory than the other routes. We'll have a handful of infrequently used routes that use >1GB while typically the monolith stays inder 200MB of ram. We'll have some expensive route that hog the even loop for a second. Mixing these routes in with the well behaved normal routes means we have to way over-provision, and means sporadic performance degredation once se lesser behaved routes start firing. Separating your stuff up is a huge win. Being able to see ingest, egress, and web traffic separately & to manage them separately is a huge win. Im not sure what the win is from a single process. I have a hard time thinking of advantages. There's some small wins to memory, but a well designed forking architecture works around most of these advantages.
- pimeys 4y ago
  I'd first wish the services would be written so that they do not use too much CPU to begin with. Ruby is horrendous with CPU usage, and the concurrency/threading in Ruby runtime is nothing to brag about. A service such as a small community Fediverse server does not need microservices and complex architecture. What I've been seeing, people are kind of struggling with Ruby and Sidekiq even when having servers that are not super big. In the past, WhatsApp was having a small group of engineers with a handful of servers, serving hundreds of millions of users with Erlang. To get people running their own Fediverse servers, it should be as easy and as cheap as possible. Akkoma is a good start, and in a few months I hope we see the first proper systems made with Rust and Go, running with the minimal amount of resources, being single process and federating everything without the user needing to do any maintenance.
  - rektide 4y ago
    I dont disagree on any point. I would still pick an implementation where I have control, management, & observability of the separate processes, such that I can see & trade off where resources are going. Mastadon aint fast, but, this current architecture has been invaluable in letting folks degrade gracefully. Sidekiq for ingress gets backed up, but often the web service is totally responsive and egress sidekiq (if seperately configured) works fine. Picking a more effective runtime might give you a lot more headroom, but I'd still want a plan & ability to cope & degrade gracefully when the system does come under pressure, and Im not sure Erlang/Beam or any other single process system is going to have such innate ability to have the different things dealt with differently. Bulkhead pattern is a truly great one- keep the whole ship from sinking when one part floods. https://learn.microsoft.com/en-us/azure/architecture/patterns/bulkhead https://learn.microsoft.com/en-us/azure/architecture/pattern...
Waterluvian 4y ago
This is the gist of what makes Django and Flask and such work fine even when you’re using some GIL-affected concurrency model like gevent. Most of the waiting for your typical web server is I/O.
INTPenis 4y ago
I need to do HTTP requests, database queries and queue processing. Mastodon has chosen a Service oriented architecture for this, and I'm happy with that decision because it means I can keep the web and DB queries running on minimal resources, while scaling the queue processing up and down depending on current activity. One could say that incoming http requests from end users is a fairly predictable level of activity, while queue processing is a very unpredictable level of activity. Because it's tied into people's posts, their followers, the number of instances their followers are on, and so forth. And with Mastodon v4 the resources required for end user http requests are even fewer, because they've shifted more processing onto the client with Javascript. So I think the freedom to be able to scale the queue processing separately is fundamental to me. I can't imagine what it would be like to run a different software where I had to scale a monolithic service. It doesn't really matter how fast the software is because queue processing is about blocking tasks, and they're blocking because they involve contact with other services and servers on the internet. So yes I'd love to see a sidekiq replacement in Golang, Rust, or even Elixir, but I would not like to see it bundled in with other services. Mainly because queue processing deals with blocking tasks, and incoming HTTP requests should never be blocking.

by pimeys 4y ago

rektide 4y ago

One of the main wins I've contributed to company after company is taking monoliths & making multiple instances of them. Yes it can run one big process of everything, and does when I find it. But some routes or subservices are slow or hog cpu or use way more memory than the other routes. We'll have a handful of infrequently used routes that use >1GB while typically the monolith stays inder 200MB of ram. We'll have some expensive route that hog the even loop for a second. Mixing these routes in with the well behaved normal routes means we have to way over-provision, and means sporadic performance degredation once se lesser behaved routes start firing. Separating your stuff up is a huge win. Being able to see ingest, egress, and web traffic separately & to manage them separately is a huge win. Im not sure what the win is from a single process. I have a hard time thinking of advantages. There's some small wins to memory, but a well designed forking architecture works around most of these advantages.

pimeys 4y ago
I'd first wish the services would be written so that they do not use too much CPU to begin with. Ruby is horrendous with CPU usage, and the concurrency/threading in Ruby runtime is nothing to brag about. A service such as a small community Fediverse server does not need microservices and complex architecture. What I've been seeing, people are kind of struggling with Ruby and Sidekiq even when having servers that are not super big. In the past, WhatsApp was having a small group of engineers with a handful of servers, serving hundreds of millions of users with Erlang. To get people running their own Fediverse servers, it should be as easy and as cheap as possible. Akkoma is a good start, and in a few months I hope we see the first proper systems made with Rust and Go, running with the minimal amount of resources, being single process and federating everything without the user needing to do any maintenance.
- rektide 4y ago
  I dont disagree on any point. I would still pick an implementation where I have control, management, & observability of the separate processes, such that I can see & trade off where resources are going. Mastadon aint fast, but, this current architecture has been invaluable in letting folks degrade gracefully. Sidekiq for ingress gets backed up, but often the web service is totally responsive and egress sidekiq (if seperately configured) works fine. Picking a more effective runtime might give you a lot more headroom, but I'd still want a plan & ability to cope & degrade gracefully when the system does come under pressure, and Im not sure Erlang/Beam or any other single process system is going to have such innate ability to have the different things dealt with differently. Bulkhead pattern is a truly great one- keep the whole ship from sinking when one part floods. https://learn.microsoft.com/en-us/azure/architecture/patterns/bulkhead https://learn.microsoft.com/en-us/azure/architecture/pattern...

Waterluvian 4y ago

This is the gist of what makes Django and Flask and such work fine even when you’re using some GIL-affected concurrency model like gevent. Most of the waiting for your typical web server is I/O.

INTPenis 4y ago

I need to do HTTP requests, database queries and queue processing. Mastodon has chosen a Service oriented architecture for this, and I'm happy with that decision because it means I can keep the web and DB queries running on minimal resources, while scaling the queue processing up and down depending on current activity. One could say that incoming http requests from end users is a fairly predictable level of activity, while queue processing is a very unpredictable level of activity. Because it's tied into people's posts, their followers, the number of instances their followers are on, and so forth. And with Mastodon v4 the resources required for end user http requests are even fewer, because they've shifted more processing onto the client with Javascript. So I think the freedom to be able to scale the queue processing separately is fundamental to me. I can't imagine what it would be like to run a different software where I had to scale a monolithic service. It doesn't really matter how fast the software is because queue processing is about blocking tasks, and they're blocking because they involve contact with other services and servers on the internet. So yes I'd love to see a sidekiq replacement in Golang, Rust, or even Elixir, but I would not like to see it bundled in with other services. Mainly because queue processing deals with blocking tasks, and incoming HTTP requests should never be blocking.