4 ms·

>Messages are always copied before being inserted into the queue. As wasteful as this may sound it greatly reduces garbage collection (GC) latency as the GC nev

>Messages are always copied before being inserted into the queue. As wasteful as this may sound it greatly reduces garbage collection (GC) latency as the GC never has to look beyond a single process. Non-copying implementations have been tried in the past, but they turned out to be a bad fit as low latency is more important than sheer throughput for the kind of soft-realtime systems that Erlang is designed to build. I do so wish there was a reference or link around "tried in the past".

dnautics 6y ago
So this isn't entirely true; binaries above a certain size have their own recounting, process independent GC going on in the BEAM; I wonder why the VM doesn't use a similar system for highly mobile data structures (copy to a common refcounted "scratchpad" on first send).
- toast0 6y ago
  It would seem off heap messages could be a base for this. I've not gone all the way into the weeds on those, but it seems like it could be possible to start with sending a whole message to another process by bumping a refcount and adding a reference to that process's mailbox without needing to make a new copy. If that works, and is useful, you could extend by bumping the refcount, and adding a new type of subreference to a process's mailbox. I don't think you'd want to send something referencing multiple off-heap messages or adding some terms and a reference (although maybe, who knows), because that would be a lot of complexity. And you would also want to be sensitive to subreference memory leaks, like can happen with binaries where a binary is large, but only small subreferences stick around, leading to lots of wasted space (this is extra fun when you put a small subbinary into ets/mnesia and it lives forever (well until you restart beam, but that can be a long time).
  - strmpnk 6y ago
    This is sort of what Pony did with its ORCA based GC. It has some tricky edge cases to optimize in practice but it can be made to work especially well if data is immutable.
- knome 6y ago
  binaries never contain references to anything else, and therefore can never be a part of a reference loop. so it's safe to toss them out into a shared space and use reference counting to track them. you don't want to do it with small binaries, as the overhead of shared-allocating/ locking/ incrementing/ locking/ decrementing would probably be worse than just coping the small binary in the first place. if you put complex reference bearing data out in the void, now whatever memory arena that allocated it may contain data that it references, and now you have to keep track of those references and use all of the shared items as new roots into your data. and if some other memory arena receives it and plucks out a value, do you only then copy it or do you reference back to the original. if you send a message back and form a loop, the gc process for either now has to jump through both, and you've destroyed your cheap gc cycles. if you shunt all of the data into shared memory to avoid multi-gc reference loops, you've now just created a big shared gc that will have all the problems of big shared gcs. the per-process small gcs with full copying between them mean you can do things like just dropping the arena when the process dies without even checking liveness for anything in it ( you'll need to run any destructors if anything needs cleanup ( like references to shared large binaries needing dereferencing ), but you can keep track of items that need that in a bitmap or something, and avoiding the trace is a win on its own )
  - KMag 6y ago
    > shared-allocating/ locking/ incrementing/ locking/ decrementing Minor nit: shared allocation, counter incrementing, and counter decrementing can all be done lock-free. They'd still need memory fence operations (and retries in case of contention), and the associated performance hits, but not actual locking.
    - fpoling 6y ago
      Lwn.net just published an article where a comment https://lwn.net/Articles/849239/ https://lwn.net/Articles/849239/ described how there were no really lock-free data structures on modern CPUs: A couple of decades of writing concurrent algorithms has taught me that scalability is really defined by the frequency the concurrent algorithm accesses/updates shared data, not whether the software algorithm is considered "lockless" or not.
- strmpnk 6y ago
  The analog for that would be shared literals which are not copied (super important for small map overhead reduction if the constructor initialized a fix set of keys like elixir structs as the key index is shared). The persistent_term module allows any value to be registered as such at runtime and avoids a lot of caveats that came with mochi_global and the like. While they aren’t ref counted it can definitely help cut down on copying in my experience.
moonchild 6y ago
Linear types (as an implementation detail) seem helpful here; if escape analysis shows a message is never touched after being sent, you can move it into the queue instead of copying it.
- joppy 6y ago
  Would this help? My understanding is that the copy was so that the new object exists in the heap of the receiving process, and therefore the GC can have a strong guarantee that all objects within a process belong to the GC for that process. Whether moving an object between two GC’d heaps is even possible (let alone simple or complicated) surely depends a lot on the GC implementation. The implementation of concurrent GCs is not easy...
dominicl 6y ago
A definitive reference would be great, for me the "tried in the past" quote has always been a pointer to the Java VM. Which has great benefits of it's own but always had the problem of global GC induced freezes. Working with Cassandra instances this has always been a problem since 2013 till today. It just seems to be a very hard problem to solve when object references flow everywhere. But it's really unpleasant when your whole DB instance with all it's parallel requests locks up for GC. [0] https://stackoverflow.com/questions/21992943/persistent-gc-issues-with-cassandra-long-app-pauses#22002767 https://stackoverflow.com/questions/21992943/persistent-gc-i...
- alexhutcheson 6y ago
  DataStax had good results testing the new Shenandoah garbage collector with Cassandra 4.0: https://www.datastax.com/blog/apache-cassandra-benchmarking-40-brings-heat-new-garbage-collectors-zgc-and-shenandoah https://www.datastax.com/blog/apache-cassandra-benchmarking-...
- secondcoming 6y ago
  Try out Scylla instead. We moved to it from Cassandra and have had no stability issues at all. Our use case is several 100k requests per second with occasional batch writes. Request latency reduced, as did the number of nodes we required.
ramchip 6y ago
There’s a few papers about it such as: - “Heap architectures for concurrent languages using message passing” - “Exploring Alternative Memory Architectures for Erlang: Implementation and Performance Evaluation” - “A Case for the Unified Heap Approach to Erlang Memory Management” You can find those and more discussion by googling shared heap erlang.
srijan4 6y ago
There's also an old discussion on this here: http://erlang.2086793.n4.nabble.com/Why-are-messages-between-processes-copied-td4675854.html http://erlang.2086793.n4.nabble.com/Why-are-messages-between...

dnautics 6y ago

So this isn't entirely true; binaries above a certain size have their own recounting, process independent GC going on in the BEAM; I wonder why the VM doesn't use a similar system for highly mobile data structures (copy to a common refcounted "scratchpad" on first send).

toast0 6y ago
It would seem off heap messages could be a base for this. I've not gone all the way into the weeds on those, but it seems like it could be possible to start with sending a whole message to another process by bumping a refcount and adding a reference to that process's mailbox without needing to make a new copy. If that works, and is useful, you could extend by bumping the refcount, and adding a new type of subreference to a process's mailbox. I don't think you'd want to send something referencing multiple off-heap messages or adding some terms and a reference (although maybe, who knows), because that would be a lot of complexity. And you would also want to be sensitive to subreference memory leaks, like can happen with binaries where a binary is large, but only small subreferences stick around, leading to lots of wasted space (this is extra fun when you put a small subbinary into ets/mnesia and it lives forever (well until you restart beam, but that can be a long time).
- strmpnk 6y ago
  This is sort of what Pony did with its ORCA based GC. It has some tricky edge cases to optimize in practice but it can be made to work especially well if data is immutable.
knome 6y ago
binaries never contain references to anything else, and therefore can never be a part of a reference loop. so it's safe to toss them out into a shared space and use reference counting to track them. you don't want to do it with small binaries, as the overhead of shared-allocating/ locking/ incrementing/ locking/ decrementing would probably be worse than just coping the small binary in the first place. if you put complex reference bearing data out in the void, now whatever memory arena that allocated it may contain data that it references, and now you have to keep track of those references and use all of the shared items as new roots into your data. and if some other memory arena receives it and plucks out a value, do you only then copy it or do you reference back to the original. if you send a message back and form a loop, the gc process for either now has to jump through both, and you've destroyed your cheap gc cycles. if you shunt all of the data into shared memory to avoid multi-gc reference loops, you've now just created a big shared gc that will have all the problems of big shared gcs. the per-process small gcs with full copying between them mean you can do things like just dropping the arena when the process dies without even checking liveness for anything in it ( you'll need to run any destructors if anything needs cleanup ( like references to shared large binaries needing dereferencing ), but you can keep track of items that need that in a bitmap or something, and avoiding the trace is a win on its own )
- KMag 6y ago
  > shared-allocating/ locking/ incrementing/ locking/ decrementing Minor nit: shared allocation, counter incrementing, and counter decrementing can all be done lock-free. They'd still need memory fence operations (and retries in case of contention), and the associated performance hits, but not actual locking.
  - fpoling 6y ago
    Lwn.net just published an article where a comment https://lwn.net/Articles/849239/ https://lwn.net/Articles/849239/ described how there were no really lock-free data structures on modern CPUs: A couple of decades of writing concurrent algorithms has taught me that scalability is really defined by the frequency the concurrent algorithm accesses/updates shared data, not whether the software algorithm is considered "lockless" or not.
strmpnk 6y ago
The analog for that would be shared literals which are not copied (super important for small map overhead reduction if the constructor initialized a fix set of keys like elixir structs as the key index is shared). The persistent_term module allows any value to be registered as such at runtime and avoids a lot of caveats that came with mochi_global and the like. While they aren’t ref counted it can definitely help cut down on copying in my experience.

moonchild 6y ago

Linear types (as an implementation detail) seem helpful here; if escape analysis shows a message is never touched after being sent, you can move it into the queue instead of copying it.

joppy 6y ago
Would this help? My understanding is that the copy was so that the new object exists in the heap of the receiving process, and therefore the GC can have a strong guarantee that all objects within a process belong to the GC for that process. Whether moving an object between two GC’d heaps is even possible (let alone simple or complicated) surely depends a lot on the GC implementation. The implementation of concurrent GCs is not easy...

dominicl 6y ago

A definitive reference would be great, for me the "tried in the past" quote has always been a pointer to the Java VM. Which has great benefits of it's own but always had the problem of global GC induced freezes. Working with Cassandra instances this has always been a problem since 2013 till today. It just seems to be a very hard problem to solve when object references flow everywhere. But it's really unpleasant when your whole DB instance with all it's parallel requests locks up for GC. [0] https://stackoverflow.com/questions/21992943/persistent-gc-issues-with-cassandra-long-app-pauses#22002767 https://stackoverflow.com/questions/21992943/persistent-gc-i...

alexhutcheson 6y ago
DataStax had good results testing the new Shenandoah garbage collector with Cassandra 4.0: https://www.datastax.com/blog/apache-cassandra-benchmarking-40-brings-heat-new-garbage-collectors-zgc-and-shenandoah https://www.datastax.com/blog/apache-cassandra-benchmarking-...
secondcoming 6y ago
Try out Scylla instead. We moved to it from Cassandra and have had no stability issues at all. Our use case is several 100k requests per second with occasional batch writes. Request latency reduced, as did the number of nodes we required.

ramchip 6y ago

There’s a few papers about it such as: - “Heap architectures for concurrent languages using message passing” - “Exploring Alternative Memory Architectures for Erlang: Implementation and Performance Evaluation” - “A Case for the Unified Heap Approach to Erlang Memory Management” You can find those and more discussion by googling shared heap erlang.

srijan4 6y ago

There's also an old discussion on this here: http://erlang.2086793.n4.nabble.com/Why-are-messages-between-processes-copied-td4675854.html http://erlang.2086793.n4.nabble.com/Why-are-messages-between...