4 ms·
I think that's a good example of how these kinds of static allocations are tricky to get right. With your scheme, you would have hundreds of millions or even bi
by tsimionescu 7y ago
I think that's a good example of how these kinds of static allocations are tricky to get right. With your scheme, you would have hundreds of millions or even billions of actors being idle all day long, and probably a few thousand actors potentially overwhelmed with processing new email every second. You would probably have to go back and decide a new partitioning scheme to avoid this, with maybe cascading consequences in other places.
The pieces are simpler, but you might pay a greater cost in system-level design. In shared state models, you could have a more dynamic approach with a number of worker threads scaling with the number of requests (emails), with the only problem being contention when accessing the same piece of the shared storage, but at least no massive overhead from idle workers. The shared-state layer would be responsible for the locking strategy, and could even potentially optimize the required locking without necessarily changing the worker code, depending on the level of abstraction it offers workers to begin with.
- toast0 7y agoWe're getting into the weeds here, so don't feel the need to reply. I agree there's different complexities, and both ways will be complex in some places, and simple in others. I just want to address this one thing. > you would have hundreds of millions or even billions of actors being idle all day long > but at least no massive overhead from idle workers. You could certainly work to activate the actor for a user only when it's needed (while the user is online, or while an email is delivered); but the level of overhead for an actor is quite low --- it would be fine to have the actor simply always running, and doing nothing when there's nothing to do. You can run a million Erlang processes on a server, and if they're mostly idle, there's no big deal; if you need to run a billion processes, you would need a thousand servers; but my wild guess is storage is going to be a bigger factor than CPU, assuming that the gmail style large quotas do get significantly used, I could easily be wrong about storage requirements though.
- tsimionescu 7y agoAgreed, we're getting in too many vague details. I didn't know that Erlang can go up to such large numbers of processes, I knew they were very lightweight, but would have guessed maybe at most thousands to tens of thousands. Very nice that they can do that!