4 ms·

Tangent: Ever since I became familiar with Erlang and the impressive BEAM, any other async method seems subpar and contrived, and that includes Python, Go, Rust

Tangent: Ever since I became familiar with Erlang and the impressive BEAM, any other async method seems subpar and contrived, and that includes Python, Go, Rust, etc. It's just weird how there's a correct way to do async and parallelism (which erlang does) and literally no other language does it.

lawn 2y ago
I'm a huge fan of the BEAM, but I wonder if you're not overselling it a little? Surely there are trade-offs here that sometimes aren't worth it and alternative ways are better? For example, the BEAM isn't optimized for throughput and if that's a high priority for you then you might want to choose something else (maybe Rust).
- behnamoh 2y ago
  > For example, the BEAM isn't optimized for throughput... weird take, given that Erlang powers telecom systems with literally millions of connections at a time.
  - deagle50 2y ago
    that's not throughput
  - jjtheblunt 2y ago
    but the traffic on each connection doesn't need bandwidth in the throughput sense mentioned
  - JackSlateur 2y ago
    I worked in telecoms, no erlang found. Perhaps we were too modern ?
    - immibis 2y ago
      I worked in telecoms. We had a third party component written in Erlang. I'd say it was about the second most reliable component of the system. It was susceptible to some memory leaks, but they usually turned out to be caused by misuse of the C client library. The most reliable component was written in C, with what must have been a space shuttle level of effort behind it. No memory allocation, except in code paths that can return an error to the user who asked for something that caused the system to need more memory (we probably got this wrong on our side of the API, and didn't test out of memory scenarios, and they probably would have just resulted in OOM kills anyway). Every line of code commented, sometimes leading to the infamous "//add 1 to i" but most times showing deep design thought. State machines documented with a paragraph for every state transition explaining non-obvious concerns.
  - toast0 2y ago
    The original use that Erlang was built for was controlling telecom switches. In that application, Erlang supervises phone lines, responding to on/off hook and dialed numbers etc, but does not touch the voice path at all --- it only controls the relays/etc to connect the voice path as desired. That's not a high throughput job at all. I used Erlang at WhatsApp. Having a million chat connections was impressive, but that's not high throughput either. Most of those connections were idle. As we added more things to the chat connection, we ended up with a significantly lower target per machine (and then at Facebook, the servers got a lot smaller and the target connections per machine was way less). We did have some Erlang machines that pushed 20gbps for https downloads, but I don't think that's that impressive; serving static files with https at 20gbps with a 2690v4 isn't hard to do if you have clients to pull the files. IMHO, Erlang is built for reliability/fault tolerance first. Process isolation happens to be good for both fault tolerance and enabling parallel processing. I find it to be a really nice environment to run lots of processes in, but it's clearly not trying to win performance prizes. If you need throughput, you need to at least process the data outside of Erlang (TLS protocol happens in Erlang, TLS crypto happens in C), and sometimes it's better to keep the data out of Erlang all together. Erlang is better suited for 'control plane' than 'data plane' applications; but it's 2024 and we have an abundance of compute power, so you can shoehorn a lot into a less than high performance environment ;)
    - behnamoh 2y ago
      Thanks, this is a really informative comment.
  - jerf 2y ago
    You need to stop reading the Erlang/BEAM propaganda from the 20th century and thinking it's still up-to-date. The BEAM ecosystem is still a viable choice today, but what you're spouting is massively out of date. BEAM is merely one viable alternative in a rich set of alternatives today. It wasn't necessarily wrong then either. I'm not saying it was "wrong". I'm saying, you're running around giving very, very old talking points. I recognize them. AIUI it's not clear that Erlang actually powers any telecoms systems anymore. Ericsson deprioritized it a long time ago. Fortunately it has found success in other niches. And I will highlight one more time, this is not a criticism of Erlang. I can, but I'm not doing that now. You're rolling into a discussion about cloud provider to use while extolling the virtues of "virtual machines", or which web framework to use while waxing poetic about these "frames" things. It's not even that you're wrong necessarily, but it's not relevant.
JackSlateur 2y ago
Could you share more intels about this ? Links or whatever I'd like to learn more
lastofus 2y ago
Other languages do sometimes implement this at the library level. Clojure's core.async comes to mind (though there are subtle differences). There's downsides to this approach though. The data going into each "mailbox" either needs to be immutable, or deep copied to be thread safe. This obviously comes at a cost. Sometimes you just have a huge amount of state that different threads need to work on, and the above solution isn't viable. Erlang has ets/dets to help deal with this. You will notice ets/dets looks nothing like the mailbox/process pattern. Erlang is great, but it is hardly the "one true way". As with most things, tradeoffs are a thing, and usually the right solution comes down to "it depends".
- wongarsu 2y ago
  > The data going into each "mailbox" either needs to be immutable, or deep copied to be thread safe Or moved. The mailbox/process pattern works great in Rust because you can simply move ownership of a value. Kind of like if in C you send the pointer and then delete your copy. Of course doing this across threads doesn't work with every type of value (what Rust's type system encodes as a value being `Send`). For example you can't send a reference-counted value if the reference counter isn't thread-safe. But that's rarely an issue and easily solved with a good type system.
OnlyMortal 2y ago
What are you talking about? In C++ you have ASIO (Boost) that’s mostly used for IPC but can be used as a general purpose async event queue. There is io_uring support too. You can sit a pool of threads as the consumers for events if you want to scale. C++ has had a defacto support for threads for ages (Boost) and it has been rolled into the standard library since 2011. If you’re using compute clusters you also have MPI in Boost. That’s a scatter/gather model. There’s also OpenMP to parallelize loops if you’re so inclined.
- bee_rider 2y ago
  I should look this up but I’m lazy. I’m familiar with MPI in Fortran/C, and IIRC they had some MPI C++ primitives a that never really got a ton of traction (that’s just the informal impression I got skimming their docs, though). How’s MPI in C++ boost work? MPI communicates big dumb arrays best I think, so maybe they did all the plumbing work for translating Boost objects into big dumb arrays, communicating those, and then reconstructing them on the other side?
  - OnlyMortal 2y ago
    It’s still a dumb way to do computation. The Boost library hides the implementation away though, on Linux and Macintosh, it requires gomp. Really, it’s a wrapper but makes it “easier” for scientists to use who are, in general, not good coders. Source: I’ve seen CERN research code.
    - bee_rider 2y ago
      What’s a dumb way to do computation? Using objects? I’m generally suspicious of divergence from the ideal form of computation (math, applied to a big dumb array) but C++ is quite popular.
mrkeen 2y ago
The actor model (share-nothing) is one way to address the problem of shared, mutable state. But what if I want to have my cake and eat it too? What if I want to have thread-safe, shared, mutable state. Is it not conceivable that there's a better approach than share-nothing?
- moffkalast 2y ago
  As much as I hate to say it, I think Java probably has the best eaten cake implementation by far. Volatile makes sure variables stay sane on the memory side, if you only write to a variable from one thread and read it from others, then it just sort of magically works? Plus the executors to handle thread reuse for async tasks. I assume C# has the same concepts given that it's just a carbon copy with title case naming. Python can't execute in on two cores at once, so it functionally has no multithreading, JS can share data between threads, but must convert it all to string because pointless performance penalties are great to have. Golang has that weird FIFO channel thing (probably sockets in disguise for these last two). C/C++ has a segfault.
  - mrkeen 2y ago
    > Volatile makes sure variables stay sane on the memory side This doesn't get you from shared-mutable-hell to shared-mutable-safe, it gets you from shared-mutable-relaxedmemorymodel-hell to shared-mutable-hell. It's the kind of hell you don't come across until you start being too smart for synchronisation primitives and start taking a stab at lockfree/lockless wizardry. > if you only write to a variable from one thread and read it from others, then it just sort of magically works I'm not necessarily convinced by that - but either way that's a huge blow to 'shared' if you are only allowed one writer. > Plus the executors to handle thread reuse for async tasks. What does this solve with regard to the shared-mutable problem? This is like "Erlang has BEAM to handle the actors" or something - so what?
    - moffkalast 2y ago
      Well it doesn't get you there because shared-mutable-safe doesn't exist, at least I doubt it can without major tradeoffs. You either err on the side of complete safety with a system that is borderline unusable for anything practical, or you let people do whatever they want and let them deal with their problems once they actually have them. > either way that's a huge blow to 'shared' if you are only allowed one writer Yeah for full N thread reading and editing you'd need N vars per var which is annoying, but that kind of every-thread-is-main setup is something that is exceedingly rare. There's almost always a few fixed main ones and lots running specific tasks that don't really need to know about all the other ones.
imtringued 2y ago
Go does something similar and ponylang is basically compiled Erlang.
Yoric 2y ago
Even Go? For me, its main redeeming point [1] was its so-called "goroutines" (which are basically Erlang lightweight processes, with the mailbox replaced with Concurrent ML-style channels). That being said, I've gotten used to Rust's async. It's a bit unusual, but it also adds a layer of clarity which I appreciate. [1] Sorry, I've really tried to click with Go, but too many things don't work with my mental models.

lawn 2y ago

I'm a huge fan of the BEAM, but I wonder if you're not overselling it a little? Surely there are trade-offs here that sometimes aren't worth it and alternative ways are better? For example, the BEAM isn't optimized for throughput and if that's a high priority for you then you might want to choose something else (maybe Rust).

behnamoh 2y ago
> For example, the BEAM isn't optimized for throughput... weird take, given that Erlang powers telecom systems with literally millions of connections at a time.
- deagle50 2y ago
  that's not throughput
- jjtheblunt 2y ago
  but the traffic on each connection doesn't need bandwidth in the throughput sense mentioned
- JackSlateur 2y ago
  I worked in telecoms, no erlang found. Perhaps we were too modern ?
  - immibis 2y ago
    I worked in telecoms. We had a third party component written in Erlang. I'd say it was about the second most reliable component of the system. It was susceptible to some memory leaks, but they usually turned out to be caused by misuse of the C client library. The most reliable component was written in C, with what must have been a space shuttle level of effort behind it. No memory allocation, except in code paths that can return an error to the user who asked for something that caused the system to need more memory (we probably got this wrong on our side of the API, and didn't test out of memory scenarios, and they probably would have just resulted in OOM kills anyway). Every line of code commented, sometimes leading to the infamous "//add 1 to i" but most times showing deep design thought. State machines documented with a paragraph for every state transition explaining non-obvious concerns.
- toast0 2y ago
  The original use that Erlang was built for was controlling telecom switches. In that application, Erlang supervises phone lines, responding to on/off hook and dialed numbers etc, but does not touch the voice path at all --- it only controls the relays/etc to connect the voice path as desired. That's not a high throughput job at all. I used Erlang at WhatsApp. Having a million chat connections was impressive, but that's not high throughput either. Most of those connections were idle. As we added more things to the chat connection, we ended up with a significantly lower target per machine (and then at Facebook, the servers got a lot smaller and the target connections per machine was way less). We did have some Erlang machines that pushed 20gbps for https downloads, but I don't think that's that impressive; serving static files with https at 20gbps with a 2690v4 isn't hard to do if you have clients to pull the files. IMHO, Erlang is built for reliability/fault tolerance first. Process isolation happens to be good for both fault tolerance and enabling parallel processing. I find it to be a really nice environment to run lots of processes in, but it's clearly not trying to win performance prizes. If you need throughput, you need to at least process the data outside of Erlang (TLS protocol happens in Erlang, TLS crypto happens in C), and sometimes it's better to keep the data out of Erlang all together. Erlang is better suited for 'control plane' than 'data plane' applications; but it's 2024 and we have an abundance of compute power, so you can shoehorn a lot into a less than high performance environment ;)
  - behnamoh 2y ago
    Thanks, this is a really informative comment.
- jerf 2y ago
  You need to stop reading the Erlang/BEAM propaganda from the 20th century and thinking it's still up-to-date. The BEAM ecosystem is still a viable choice today, but what you're spouting is massively out of date. BEAM is merely one viable alternative in a rich set of alternatives today. It wasn't necessarily wrong then either. I'm not saying it was "wrong". I'm saying, you're running around giving very, very old talking points. I recognize them. AIUI it's not clear that Erlang actually powers any telecoms systems anymore. Ericsson deprioritized it a long time ago. Fortunately it has found success in other niches. And I will highlight one more time, this is not a criticism of Erlang. I can, but I'm not doing that now. You're rolling into a discussion about cloud provider to use while extolling the virtues of "virtual machines", or which web framework to use while waxing poetic about these "frames" things. It's not even that you're wrong necessarily, but it's not relevant.

JackSlateur 2y ago

Could you share more intels about this ? Links or whatever I'd like to learn more

lastofus 2y ago

Other languages do sometimes implement this at the library level. Clojure's core.async comes to mind (though there are subtle differences). There's downsides to this approach though. The data going into each "mailbox" either needs to be immutable, or deep copied to be thread safe. This obviously comes at a cost. Sometimes you just have a huge amount of state that different threads need to work on, and the above solution isn't viable. Erlang has ets/dets to help deal with this. You will notice ets/dets looks nothing like the mailbox/process pattern. Erlang is great, but it is hardly the "one true way". As with most things, tradeoffs are a thing, and usually the right solution comes down to "it depends".

wongarsu 2y ago
> The data going into each "mailbox" either needs to be immutable, or deep copied to be thread safe Or moved. The mailbox/process pattern works great in Rust because you can simply move ownership of a value. Kind of like if in C you send the pointer and then delete your copy. Of course doing this across threads doesn't work with every type of value (what Rust's type system encodes as a value being `Send`). For example you can't send a reference-counted value if the reference counter isn't thread-safe. But that's rarely an issue and easily solved with a good type system.

OnlyMortal 2y ago

What are you talking about? In C++ you have ASIO (Boost) that’s mostly used for IPC but can be used as a general purpose async event queue. There is io_uring support too. You can sit a pool of threads as the consumers for events if you want to scale. C++ has had a defacto support for threads for ages (Boost) and it has been rolled into the standard library since 2011. If you’re using compute clusters you also have MPI in Boost. That’s a scatter/gather model. There’s also OpenMP to parallelize loops if you’re so inclined.

bee_rider 2y ago
I should look this up but I’m lazy. I’m familiar with MPI in Fortran/C, and IIRC they had some MPI C++ primitives a that never really got a ton of traction (that’s just the informal impression I got skimming their docs, though). How’s MPI in C++ boost work? MPI communicates big dumb arrays best I think, so maybe they did all the plumbing work for translating Boost objects into big dumb arrays, communicating those, and then reconstructing them on the other side?
- OnlyMortal 2y ago
  It’s still a dumb way to do computation. The Boost library hides the implementation away though, on Linux and Macintosh, it requires gomp. Really, it’s a wrapper but makes it “easier” for scientists to use who are, in general, not good coders. Source: I’ve seen CERN research code.
  - bee_rider 2y ago
    What’s a dumb way to do computation? Using objects? I’m generally suspicious of divergence from the ideal form of computation (math, applied to a big dumb array) but C++ is quite popular.

mrkeen 2y ago

The actor model (share-nothing) is one way to address the problem of shared, mutable state. But what if I want to have my cake and eat it too? What if I want to have thread-safe, shared, mutable state. Is it not conceivable that there's a better approach than share-nothing?

moffkalast 2y ago
As much as I hate to say it, I think Java probably has the best eaten cake implementation by far. Volatile makes sure variables stay sane on the memory side, if you only write to a variable from one thread and read it from others, then it just sort of magically works? Plus the executors to handle thread reuse for async tasks. I assume C# has the same concepts given that it's just a carbon copy with title case naming. Python can't execute in on two cores at once, so it functionally has no multithreading, JS can share data between threads, but must convert it all to string because pointless performance penalties are great to have. Golang has that weird FIFO channel thing (probably sockets in disguise for these last two). C/C++ has a segfault.
- mrkeen 2y ago
  > Volatile makes sure variables stay sane on the memory side This doesn't get you from shared-mutable-hell to shared-mutable-safe, it gets you from shared-mutable-relaxedmemorymodel-hell to shared-mutable-hell. It's the kind of hell you don't come across until you start being too smart for synchronisation primitives and start taking a stab at lockfree/lockless wizardry. > if you only write to a variable from one thread and read it from others, then it just sort of magically works I'm not necessarily convinced by that - but either way that's a huge blow to 'shared' if you are only allowed one writer. > Plus the executors to handle thread reuse for async tasks. What does this solve with regard to the shared-mutable problem? This is like "Erlang has BEAM to handle the actors" or something - so what?
  - moffkalast 2y ago
    Well it doesn't get you there because shared-mutable-safe doesn't exist, at least I doubt it can without major tradeoffs. You either err on the side of complete safety with a system that is borderline unusable for anything practical, or you let people do whatever they want and let them deal with their problems once they actually have them. > either way that's a huge blow to 'shared' if you are only allowed one writer Yeah for full N thread reading and editing you'd need N vars per var which is annoying, but that kind of every-thread-is-main setup is something that is exceedingly rare. There's almost always a few fixed main ones and lots running specific tasks that don't really need to know about all the other ones.

imtringued 2y ago

Go does something similar and ponylang is basically compiled Erlang.

Yoric 2y ago

Even Go? For me, its main redeeming point [1] was its so-called "goroutines" (which are basically Erlang lightweight processes, with the mailbox replaced with Concurrent ML-style channels). That being said, I've gotten used to Rust's async. It's a bit unusual, but it also adds a layer of clarity which I appreciate. [1] Sorry, I've really tried to click with Go, but too many things don't work with my mental models.