4 ms·

Best Erlang design decision: copying messages between process heaps. Con: a small performance cost. Pro: a far simpler system in every other aspect. Garbage Co

by jlouis 9y ago

Best Erlang design decision: copying messages between process heaps. Con: a small performance cost. Pro: a far simpler system in every other aspect. Garbage Collection and ownership is much easier when you are making copies of data.

amelius 9y ago
> Con: a small performance cost. That entirely depends on the amount of data that needs to be copied (!) In many cases, it pays off to actually use that intricate shared memory system when possible. Yes, you lose some predictability, but you gain a lot of performance.
- SEMW 9y ago
  Note that binaries >64 bytes are indeed stored in shared memory and reference counted. (Including utf8 strings (as opposed to charlists), which are binaries in erlang). See http://erlang.org/doc/efficiency_guide/binaryhandling.html#refc_binary http://erlang.org/doc/efficiency_guide/binaryhandling.html#r... for more info
- jlouis 9y ago
  In Erlang programs, the amount of data to be copied tends to be rather small. You'd much rather bring the computation to the data than sending the data around for computation. If you are storing a lot of data, you either tend to keep it in ETS, which has its own storage engine (and you copy the small parts of data you work on in/out of the ETS system), or as binary data, which are tracked by an ARC-like system in its own arena/heap. Of course, if you systematically abuse the system and ask it to copy several megabytes around all the time, your efficiency will suffer. But note that this is bad form: yes, it is faster locally internally inside a given node, but once you start running true distribution, you don't want to copy megabytes of data between nodes anyway. Shared memory tend to imply a lot of locks, which can bring a lot of contention as well. It is far more complex than just a general rule which says "copying is always bad". And this was my main point, more or less. It is a mistake to see copying for all its bad behaviour, without ever wondering what it brings to the table as a result.
  - amelius 9y ago
    I'm not saying copying is always bad. I'm saying that there are cases where you really want sharing, and therefore your language should allow it.
rdtsc 9y ago
Because of immutable data, it is really an implementation detail. So it would be possible to switch implementation and with extra complexity have it where messages are not copied. I even seem to recall there was a historical implementation of the Erlang VM which didn't copy the messages, or maybe it was a port to an alternate VM like JVM...

by jlouis 9y ago

amelius 9y ago

> Con: a small performance cost. That entirely depends on the amount of data that needs to be copied (!) In many cases, it pays off to actually use that intricate shared memory system when possible. Yes, you lose some predictability, but you gain a lot of performance.

SEMW 9y ago
Note that binaries >64 bytes are indeed stored in shared memory and reference counted. (Including utf8 strings (as opposed to charlists), which are binaries in erlang). See http://erlang.org/doc/efficiency_guide/binaryhandling.html#refc_binary http://erlang.org/doc/efficiency_guide/binaryhandling.html#r... for more info
jlouis 9y ago
In Erlang programs, the amount of data to be copied tends to be rather small. You'd much rather bring the computation to the data than sending the data around for computation. If you are storing a lot of data, you either tend to keep it in ETS, which has its own storage engine (and you copy the small parts of data you work on in/out of the ETS system), or as binary data, which are tracked by an ARC-like system in its own arena/heap. Of course, if you systematically abuse the system and ask it to copy several megabytes around all the time, your efficiency will suffer. But note that this is bad form: yes, it is faster locally internally inside a given node, but once you start running true distribution, you don't want to copy megabytes of data between nodes anyway. Shared memory tend to imply a lot of locks, which can bring a lot of contention as well. It is far more complex than just a general rule which says "copying is always bad". And this was my main point, more or less. It is a mistake to see copying for all its bad behaviour, without ever wondering what it brings to the table as a result.
- amelius 9y ago
  I'm not saying copying is always bad. I'm saying that there are cases where you really want sharing, and therefore your language should allow it.

rdtsc 9y ago

Because of immutable data, it is really an implementation detail. So it would be possible to switch implementation and with extra complexity have it where messages are not copied. I even seem to recall there was a historical implementation of the Erlang VM which didn't copy the messages, or maybe it was a port to an alternate VM like JVM...