3 ms·

For some reason, GC is on of those areas that I have zero desire to learn sufficiently to tune. That's probably why OP has a successful career as a consultant.

For some reason, GC is on of those areas that I have zero desire to learn sufficiently to tune. That's probably why OP has a successful career as a consultant. When making tradeoffs in development, a simpler GC with predictable behavior is worth losing a lot of raw performance to me. Thus I find myself drawn to platforms like erlang/elixir or Haskell where GC is isolated. I suppose the rational approach is build on Hotspot and when it becomes a problem hire a consultant for instant performance gains (assuming low hanging fruit is in abundance).

pron 11y ago
In 80% of the cases, simple GC ergonomics (i.e. let HotSpot figure out the tuning) are more than ok, and in 95% of the cases, trivial tuning is enough. The thing with Erlang is that the GC doesn't work on any shared memory data structure (like ETS), so you pretty much have to delegate any shared data to an out-of-process database, even in simple cases that are easily addressed with ConcurrentHashMap, ConcurrentSkipListMap/Set, or CopyOnWriteArrayList/Set on the JVM.
- phamilton 11y ago
  I generally use an Agent in Elixir for anything shared and there's very little cognitive overhead (though perf does take a hit). Agent.cast/2 is helpful for low latency modifications (at the cost of losing back pressure). As stated, it's a tradeoff.
  - pron 11y ago
    But, if I'm not mistaken, an agent isn't concurrent. You make your shared data a bottleneck which is exactly what concurrent data strctcures are meant to prevent.
- duaneb 11y ago
  Shared memory data structures are rarely necessary. When one is, I would be willing to go the extra mile to facilitate it. Seems like a reasonable tradeoff.
- rdtsc 11y ago
  Good point, but ETS is still managed and owned by a process. So it still can be cleaned up taken care of. Moreover at the API level ETS even behaves as if there is a process that manages data (even though implementation-wise it is all done in C). Also in Erlang large binaries are not copied but shared as well. And whether large shared data-structures are needed is dependent on the application. I imagine since you've done work with spatial relationships, in that case for anything interesting to happen data has to be indexed, shared, queried in one place. But say building a large concurrent chat or messaging application with many but small state machine actor that might have lots of peer to peer interaction, then shared data is not as crucial.
rdtsc 11y ago
Yeah that is why I like the BEAM VM as well. It is a really awesome piece of engineering build for massive concurrency and fault tolerance. It is nice to see the whole Elixir ecosystem take advantage of it. That model of concurrency is important to reduce cognitive load on the programmer. Having concurrent units and mutable state isolated makes it much easier to understand what is going on, rather than having some large class or shared memory state modified from many code paths concurrently (maybe via callbacks, threads, signals, etc.)

pron 11y ago

In 80% of the cases, simple GC ergonomics (i.e. let HotSpot figure out the tuning) are more than ok, and in 95% of the cases, trivial tuning is enough. The thing with Erlang is that the GC doesn't work on any shared memory data structure (like ETS), so you pretty much have to delegate any shared data to an out-of-process database, even in simple cases that are easily addressed with ConcurrentHashMap, ConcurrentSkipListMap/Set, or CopyOnWriteArrayList/Set on the JVM.

phamilton 11y ago
I generally use an Agent in Elixir for anything shared and there's very little cognitive overhead (though perf does take a hit). Agent.cast/2 is helpful for low latency modifications (at the cost of losing back pressure). As stated, it's a tradeoff.
- pron 11y ago
  But, if I'm not mistaken, an agent isn't concurrent. You make your shared data a bottleneck which is exactly what concurrent data strctcures are meant to prevent.
duaneb 11y ago
Shared memory data structures are rarely necessary. When one is, I would be willing to go the extra mile to facilitate it. Seems like a reasonable tradeoff.
rdtsc 11y ago
Good point, but ETS is still managed and owned by a process. So it still can be cleaned up taken care of. Moreover at the API level ETS even behaves as if there is a process that manages data (even though implementation-wise it is all done in C). Also in Erlang large binaries are not copied but shared as well. And whether large shared data-structures are needed is dependent on the application. I imagine since you've done work with spatial relationships, in that case for anything interesting to happen data has to be indexed, shared, queried in one place. But say building a large concurrent chat or messaging application with many but small state machine actor that might have lots of peer to peer interaction, then shared data is not as crucial.

rdtsc 11y ago

Yeah that is why I like the BEAM VM as well. It is a really awesome piece of engineering build for massive concurrency and fault tolerance. It is nice to see the whole Elixir ecosystem take advantage of it. That model of concurrency is important to reduce cognitive load on the programmer. Having concurrent units and mutable state isolated makes it much easier to understand what is going on, rather than having some large class or shared memory state modified from many code paths concurrently (maybe via callbacks, threads, signals, etc.)