4 ms·

> The confused look told me I was asking an odd question: processes in BEAM are as isolated from each other as console apps running in different VMs in VMWare s

> The confused look told me I was asking an odd question: processes in BEAM are as isolated from each other as console apps running in different VMs in VMWare server. These processes can send messages to each other but they cannot access each other's memory. It was truly a mind-blowing revelation! Not quite: the messages sent are kept in shared memory (which is reference-counted). Cycles can't occur due to immutability. But with everything else, you're correct. One reason for this, is that every thread has its own garbage collector! This allows multithreaded garbage collection without stop-the-world pauses. Source: Did a bunch of research into BEAM for implementing my own multithreaded GC. BEAM really is beautiful!

vvanders 7y ago
> This allows multithreaded garbage collection without stop-the-world pauses. This was one of the things that really floored me when I learned about how BEAM was designed. It's incredible that how all the seemingly small changes come together in a way that works really well for the problem domains Erlang was used for.
- arthurcolle 7y ago
  Anyone interested in this subject would be extremely well-served by picking up a copy of the late, great Joe Armstrong's "Programming Erlang." Putting together a minimal "server" and then slowly evolving it into a GenServer is incredibly fun and rewarding. And then the kicker is you get hot-swapping for (almost) free! Learning about Erlang hot-swapping was probably one of my most favorite moments in my brief time using Erlang before moving onto Elixir, where you get it all right out of the box. BEAM is truly a modern engineering marvel.
- macintux 7y ago
  This is exactly the theme of my favorite talk. Erlang has some cool features, but it's the way they enhance each other that's amazing. https://youtu.be/E18shi1qIHU https://youtu.be/E18shi1qIHU
ramchip 7y ago
> the messages sent are kept in shared memory (which is reference-counted) Only large binaries and slices of them use reference counting, other things are copied. And of course, if the process is remote, everything is copied :)
- toast0 7y ago
  Not all refcounted binaries are large btw. It's pretty easy to construct a refc binary [1] Bin0 = <<0>>, %% 1 Bin1 = <<Bin0/binary,1,2,3>>, %% 2 Bin0 was on the process heap, but Bin1 is a refcounted binary with 256 bytes of storage. If you're putting Bin1 into ets/mnesia, use binary:copy/1 to get a clean copy that doesn't reference a lot of extra storage. [1] http://www1.erlang.org/doc/efficiency_guide/binaryhandling.html#id65352 http://www1.erlang.org/doc/efficiency_guide/binaryhandling.h...
- kerkeslager 7y ago
  Interesting. I'm glad to be wrong because then I learn something! I'm not sure if you have answers, but I have questions: 1. How do they determine what size to start sharing/refcounting at? Now that I'm actually considering it, it's obvious that they'd copy small stuff--an obvious lower limit would be sizeof(<reference_counter_type>) + sizeof(<pointer_type>) because smaller than that, it actually is faster/less memory to copy. But the numbers you're mentioning are higher than that--is there some metadata being stored beyond the reference count and pointer? 2. Is copied data stored in thread storage and garbage collected by that thread's garbage collector? That would be my guess. Obviously the source of truth for all these questions is the code[1], but it's been a while since I dug in there. Maybe it's time to go again. [1] https://github.com/erlang/otp https://github.com/erlang/otp
  - elcritch 7y ago
    Regarding question 1, my understanding is that all non-binaries messages are copied regardless of size. Only binaries ever use shared memory. Early in BEAM development they debated optimizing shared memory for local messages but ended up not using it, partly due to the extra code complexity. Always copying messages means there’s little difference between local or remote messaging code paths and removes ref counting. Haven’t read the source myself regarding that, but it would be an interesting price to study. I’d bet optimizing local message passing would be a good feature of a BEAM implementation in Rust since the ref counting and other details would be less worrisome. Another research topic would be how BEAM would compare to say Java/Dotnet runtimes on cpu cores with lessened hardware shared memory guarantees and relying on message copying between cores instead.

vvanders 7y ago

> This allows multithreaded garbage collection without stop-the-world pauses. This was one of the things that really floored me when I learned about how BEAM was designed. It's incredible that how all the seemingly small changes come together in a way that works really well for the problem domains Erlang was used for.

arthurcolle 7y ago
Anyone interested in this subject would be extremely well-served by picking up a copy of the late, great Joe Armstrong's "Programming Erlang." Putting together a minimal "server" and then slowly evolving it into a GenServer is incredibly fun and rewarding. And then the kicker is you get hot-swapping for (almost) free! Learning about Erlang hot-swapping was probably one of my most favorite moments in my brief time using Erlang before moving onto Elixir, where you get it all right out of the box. BEAM is truly a modern engineering marvel.
macintux 7y ago
This is exactly the theme of my favorite talk. Erlang has some cool features, but it's the way they enhance each other that's amazing. https://youtu.be/E18shi1qIHU https://youtu.be/E18shi1qIHU

ramchip 7y ago

> the messages sent are kept in shared memory (which is reference-counted) Only large binaries and slices of them use reference counting, other things are copied. And of course, if the process is remote, everything is copied :)

toast0 7y ago
Not all refcounted binaries are large btw. It's pretty easy to construct a refc binary [1] Bin0 = <<0>>, %% 1 Bin1 = <<Bin0/binary,1,2,3>>, %% 2 Bin0 was on the process heap, but Bin1 is a refcounted binary with 256 bytes of storage. If you're putting Bin1 into ets/mnesia, use binary:copy/1 to get a clean copy that doesn't reference a lot of extra storage. [1] http://www1.erlang.org/doc/efficiency_guide/binaryhandling.html#id65352 http://www1.erlang.org/doc/efficiency_guide/binaryhandling.h...
kerkeslager 7y ago
Interesting. I'm glad to be wrong because then I learn something! I'm not sure if you have answers, but I have questions: 1. How do they determine what size to start sharing/refcounting at? Now that I'm actually considering it, it's obvious that they'd copy small stuff--an obvious lower limit would be sizeof(<reference_counter_type>) + sizeof(<pointer_type>) because smaller than that, it actually is faster/less memory to copy. But the numbers you're mentioning are higher than that--is there some metadata being stored beyond the reference count and pointer? 2. Is copied data stored in thread storage and garbage collected by that thread's garbage collector? That would be my guess. Obviously the source of truth for all these questions is the code[1], but it's been a while since I dug in there. Maybe it's time to go again. [1] https://github.com/erlang/otp https://github.com/erlang/otp
- elcritch 7y ago
  Regarding question 1, my understanding is that all non-binaries messages are copied regardless of size. Only binaries ever use shared memory. Early in BEAM development they debated optimizing shared memory for local messages but ended up not using it, partly due to the extra code complexity. Always copying messages means there’s little difference between local or remote messaging code paths and removes ref counting. Haven’t read the source myself regarding that, but it would be an interesting price to study. I’d bet optimizing local message passing would be a good feature of a BEAM implementation in Rust since the ref counting and other details would be less worrisome. Another research topic would be how BEAM would compare to say Java/Dotnet runtimes on cpu cores with lessened hardware shared memory guarantees and relying on message copying between cores instead.