4 ms·

> Sending messages between Erlang processes was not as cheap as we expected, and the reduction cost — Erlang unit of work used for process scheduling — was also

> Sending messages between Erlang processes was not as cheap as we expected, and the reduction cost — Erlang unit of work used for process scheduling — was also quite high. This is really surprising to me, and definitely something the elixir team should look at optimizing. Sending messages should be extremely fast.

zinclozenge 8y ago
That's something that the BEAM group would work on, although jose has contributed some patches to the compiler.
jadbox 8y ago
I have no insight into how Erlang performs this process, but I'd assume Erlang is performing sending messages almost as fast as it's possible, as it's a key part of the platform and has had two decades to perfect it. Most likely this is a limitation of the paradigm (of the Actor model) and not the implementation.
- larryweya 8y ago
  I believe the bottleneck the author was trying to overcome here is how the Erlang VM moves processes to the back of the run queue once they hit a predetermined number of operations (preemptive scheduling) and not of the Actor Model.
keithnoizu 8y ago
There are ways to speed up message passing. It's not slow by any means but weird things become your bottle neck when you have 5m concurrent users. For instance you have to build you OTP tree extra layers deep or spawn hundreds of supervisors because just routine calls to the master process to add / remove children begins to become bottle neck. In my implementation, only 1/2 a million concurrent IOT devices, I use routing tables that narrow down a process to a node and registry cluster (https://hexdocs.pm/elixir/master/Registry.html https://hexdocs.pm/elixir/master/Registry.html) , and from that registry it fans out to 1 of 100 supervisors for that worker type per node.
jhgg 8y ago
The high reduction cost is deliberate. Reductions in BEAM are an arbitrary value that certain operations in ERTS assigned. Sending to a remote node has a high reduction cost, which means that the more sends you do, the more your process has to be rescheduled to do remote sends. We worked around this with manifold, by limiting the number of remote sends we do to fan-out messages, and transforming them into local sends on the receiving nodes, which has a much cheaper "reduction" cost.
apabepa 8y ago
If the message is sent to a remote host then perhaps the send_no_suspend function could help. I know it is used in core parts of erlang telcom systems with several millions of users to work around performance issues http://erlang.org/doc/man/erlang.html#send_nosuspend-2 http://erlang.org/doc/man/erlang.html#send_nosuspend-2

zinclozenge 8y ago

That's something that the BEAM group would work on, although jose has contributed some patches to the compiler.

jadbox 8y ago

I have no insight into how Erlang performs this process, but I'd assume Erlang is performing sending messages almost as fast as it's possible, as it's a key part of the platform and has had two decades to perfect it. Most likely this is a limitation of the paradigm (of the Actor model) and not the implementation.

larryweya 8y ago
I believe the bottleneck the author was trying to overcome here is how the Erlang VM moves processes to the back of the run queue once they hit a predetermined number of operations (preemptive scheduling) and not of the Actor Model.

keithnoizu 8y ago

There are ways to speed up message passing. It's not slow by any means but weird things become your bottle neck when you have 5m concurrent users. For instance you have to build you OTP tree extra layers deep or spawn hundreds of supervisors because just routine calls to the master process to add / remove children begins to become bottle neck. In my implementation, only 1/2 a million concurrent IOT devices, I use routing tables that narrow down a process to a node and registry cluster (https://hexdocs.pm/elixir/master/Registry.html https://hexdocs.pm/elixir/master/Registry.html) , and from that registry it fans out to 1 of 100 supervisors for that worker type per node.

jhgg 8y ago

The high reduction cost is deliberate. Reductions in BEAM are an arbitrary value that certain operations in ERTS assigned. Sending to a remote node has a high reduction cost, which means that the more sends you do, the more your process has to be rescheduled to do remote sends. We worked around this with manifold, by limiting the number of remote sends we do to fan-out messages, and transforming them into local sends on the receiving nodes, which has a much cheaper "reduction" cost.

apabepa 8y ago

If the message is sent to a remote host then perhaps the send_no_suspend function could help. I know it is used in core parts of erlang telcom systems with several millions of users to work around performance issues http://erlang.org/doc/man/erlang.html#send_nosuspend-2 http://erlang.org/doc/man/erlang.html#send_nosuspend-2