4 ms·

All interesting replies, but as I am not an erlang programmer could someone briefly explain the threading model? Whatever the unit of code packaging is ultimate

by markbnj 11y ago

All interesting replies, but as I am not an erlang programmer could someone briefly explain the threading model? Whatever the unit of code packaging is ultimately it all has to get scheduled onto a core. On what platform is 2,000,000 threads a realistic scenario? If you don't have 2,000,000 threads you don't have 2,000,000 web servers. You might, for example, have 2000 web servers each using an event-driven framework to handle 1000 concurrent requests. Is that closer to the erlang processing model?

phamilton 11y ago
A single preemptive event driven framework is probably a closer match. BEAM has a very efficient multi core fair scheduler. All 2M process share time on all available cores and are handled by a single scheduler. Because it's preemptive, starvation problems one might see on an event loop are avoided. A CPU intensive process can run alongside low latency IO processes and they are all fairly scheduled.
rakoo 11y ago
As others have said, Erlang processes are not OS processes, nor are they Java threads. They're closer to Go's goroutines or Python's eventlets; they are well-suited to run even a single function concurrently. In the analogy, it's like having 2M web servers, but where only, say, 1000 of them are actually running. As a developer having independent web servers helps you to think about what should be done instead of how it's going to be done. After that, the fact that a server is actually running or it is stalled because it's reading from disk or it is waiting for available CPU is an implementation detail and something that monitoring will show you.
toast0 11y ago
The erlang process model, since SMP support is more or less, each cpu core runs one OS thread which is running a 'scheduler'. Each scheduler has a list of processes, and determines the order they should get to run in; the currently scheduled process is run in the scheduler's thread, until it has used its quota of cpu (reductions) or it hits some other trigger that causes its execution to be suspended (waiting for a message, sending a message to a busy port, maybe some other cases?). Schedulers have a method to balance their work queues, etc. (Leaving out dirty schedulers, io schedulers, configuration of schedulers etc) Each erlang process has its own (erlang) heap and stack, and there is no ability within the Erlang language to access memory by address, a process will only access its own memory, or special types of shared memory (such as the shared/large binary heap). From the OS perspective, it's one process, with one memory space, and several threads. A realistic Erlang example where you would have 2 Million processes is something like a chat server (1 process per connected user), either via a traditional tcp protocol or a websocket protocol for a browser. In another language, you would likely need to multiplex multiple users onto each thread, in Erlang each user gets their own process and the code ends up being quite straight forward.

by markbnj 11y ago

phamilton 11y ago

A single preemptive event driven framework is probably a closer match. BEAM has a very efficient multi core fair scheduler. All 2M process share time on all available cores and are handled by a single scheduler. Because it's preemptive, starvation problems one might see on an event loop are avoided. A CPU intensive process can run alongside low latency IO processes and they are all fairly scheduled.

rakoo 11y ago

As others have said, Erlang processes are not OS processes, nor are they Java threads. They're closer to Go's goroutines or Python's eventlets; they are well-suited to run even a single function concurrently. In the analogy, it's like having 2M web servers, but where only, say, 1000 of them are actually running. As a developer having independent web servers helps you to think about what should be done instead of how it's going to be done. After that, the fact that a server is actually running or it is stalled because it's reading from disk or it is waiting for available CPU is an implementation detail and something that monitoring will show you.

toast0 11y ago

The erlang process model, since SMP support is more or less, each cpu core runs one OS thread which is running a 'scheduler'. Each scheduler has a list of processes, and determines the order they should get to run in; the currently scheduled process is run in the scheduler's thread, until it has used its quota of cpu (reductions) or it hits some other trigger that causes its execution to be suspended (waiting for a message, sending a message to a busy port, maybe some other cases?). Schedulers have a method to balance their work queues, etc. (Leaving out dirty schedulers, io schedulers, configuration of schedulers etc) Each erlang process has its own (erlang) heap and stack, and there is no ability within the Erlang language to access memory by address, a process will only access its own memory, or special types of shared memory (such as the shared/large binary heap). From the OS perspective, it's one process, with one memory space, and several threads. A realistic Erlang example where you would have 2 Million processes is something like a chat server (1 process per connected user), either via a traditional tcp protocol or a websocket protocol for a browser. In another language, you would likely need to multiplex multiple users onto each thread, in Erlang each user gets their own process and the code ends up being quite straight forward.