4 ms·

There’s a lot of literature on this in the Erlang world. This post is a very thorough summary of the various solutions available: https://ferd.ca/handling-over

There’s a lot of literature on this in the Erlang world. This post is a very thorough summary of the various solutions available: https://ferd.ca/handling-overload.html https://ferd.ca/handling-overload.html I don’t find that kind of fully async processing pipeline very idiomatic TBH, at least in Erlang - more typically, every request would be its own actor, and do blocking requests to other actors that own shared resources.

zmmmmm 9y ago
Thank you for the insightful comments. That link looks perfect (down to the point of having a diagram with a->b->c->d just like mine!). The idea of each request being its own actor being more idiomatic is intriguing. Naively though does it really solve the problem? Don't you just end up with an overflow of actors now instead of an overflow of one actor's inbox?
- lliamander 9y ago
  In Erlang the actors are lightweight processes are much smaller than OS threads. I wouldn't worry.
  - zmmmmm 9y ago
    Right, but in my scenario (and I appreciate perhaps this means I'm "doing it wrong"), the messages themselves have a fairly heavy payload (around 800 bytes). Given that, whether I'm spinning up an actor or a message, if it can't be processed quickly it's consuming a non trivial chunk of memory.
    - lliamander 9y ago
      I wouldn't say you are doing anything wrong (or, at least I couldn't say that for certain without knowing more about your problem domain :) ). However, I also don't think you have to worry about the message payloads being too large. In Erlang, normally objects are copied between processes. However, for large objects (>64 bytes) they are put in a shared heap, and only the pointers are copied between processes[0]. [0] https://hamidreza-s.github.io/erlang%20garbage%20collection%20memory%20layout%20soft%20realtime/2015/08/24/erlang-garbage-collection-details-and-why-it-matters.html https://hamidreza-s.github.io/erlang%20garbage%20collection%...
- ramchip 9y ago
  You can put a bound on the number of actors under a given supervisor. That being said the main reason an Erlang program will use an actor per request is not to prevent overload, it’s for fault tolerance; in your a -> b -> c -> d example a bug (e.g. uncaught exception) triggered by one request that happens in process c can cause all requests in flight to fail, while with a process per request there’s only one affected. It does mean you may have thousands of actors rather than four or so, but that’s not a problem since Erlang is built and optimized for that kind of workload. On a platform where actors are OS threads, it may make sense to use a different approach.

zmmmmm 9y ago

Thank you for the insightful comments. That link looks perfect (down to the point of having a diagram with a->b->c->d just like mine!). The idea of each request being its own actor being more idiomatic is intriguing. Naively though does it really solve the problem? Don't you just end up with an overflow of actors now instead of an overflow of one actor's inbox?

lliamander 9y ago
In Erlang the actors are lightweight processes are much smaller than OS threads. I wouldn't worry.
- zmmmmm 9y ago
  Right, but in my scenario (and I appreciate perhaps this means I'm "doing it wrong"), the messages themselves have a fairly heavy payload (around 800 bytes). Given that, whether I'm spinning up an actor or a message, if it can't be processed quickly it's consuming a non trivial chunk of memory.
  - lliamander 9y ago
    I wouldn't say you are doing anything wrong (or, at least I couldn't say that for certain without knowing more about your problem domain :) ). However, I also don't think you have to worry about the message payloads being too large. In Erlang, normally objects are copied between processes. However, for large objects (>64 bytes) they are put in a shared heap, and only the pointers are copied between processes[0]. [0] https://hamidreza-s.github.io/erlang%20garbage%20collection%20memory%20layout%20soft%20realtime/2015/08/24/erlang-garbage-collection-details-and-why-it-matters.html https://hamidreza-s.github.io/erlang%20garbage%20collection%...
ramchip 9y ago
You can put a bound on the number of actors under a given supervisor. That being said the main reason an Erlang program will use an actor per request is not to prevent overload, it’s for fault tolerance; in your a -> b -> c -> d example a bug (e.g. uncaught exception) triggered by one request that happens in process c can cause all requests in flight to fail, while with a process per request there’s only one affected. It does mean you may have thousands of actors rather than four or so, but that’s not a problem since Erlang is built and optimized for that kind of workload. On a platform where actors are OS threads, it may make sense to use a different approach.