7 ms·

"This begs the question: why aren’t language and library designers stealing the structure behind Erlang’s behaviours, rather than copying the ideas of lightweig

by jerf 2y ago

"This begs the question: why aren’t language and library designers stealing the structure behind Erlang’s behaviours, rather than copying the ideas of lightweight processes and message passing?" Because the function signatures of Erlang's behaviors are critically tied to Erlang's other functionality, specifically its unusual use of immutability. You need a separate init call for its servers because of that, and a very distinct use of the state management to work exactly the same way. But to achieve the same goals in other languages, you almost always shouldn't directly copy what Erlang is doing. In fact when I see "Look! I ported gen_server into $SOME_OTHER_LANGUAGE" and I see exactly and precisely the exact interface Erlang has, I know that the port doesn't deeply understand what Erlang is doing. When I ported the idea of supervisor trees into Go [1], I did so idiomatically. It turns out in modern Go the correct interface for "a thing that can be supervised" is not precisely the same signature that Erlang has, but type Service interface { Serve(context.Context) } That's all you need and all you should use... in Go. Your other language may vary. Go doesn't need a "handle_event/2" because it has channels, and you should use those, not because they are "better" or "worse" but because that's what this language does. In another language you may use something else. In another infrastructure you may end up sending things over Kafka or some cloud event bus rather than "calling a handle_event/2". The key is in building an event-based system, not copying the exact implementation Erlang has. A peculiar issue the Erlang community has is getting excessively convinced that there's something super-mega-special about the exact way Erlang does it, and that if you do it any other way it is ipso facto wrong and therefore not reliable. This may have been true in 2005; it is not true in 2025. Where once Erlang had almost the only sensible answer, in 2025 the problem is poking through the ocean of answers deluging us! While I recommend learning from Erlang about reliable software, I strongly recommend against just blind-porting out the exact way Erlang achieves it into any other language. It is in almost any other language context the wrong answer. Even other immutable languages generally vary enough that they can't just copy the same structure. [1]: https://jerf.org/iri/post/2930/ https://jerf.org/iri/post/2930/

asa400 2y ago
To follow on from your excellent post, I think a reasonable next question is, "why have these kinds of approaches and ideas in other languages and systems succeeded in gaining market adoption, but Erlang/Elixir has not?" This to me is the most interesting question about Erlang, and I say this as someone who works professionally in Elixir. It's _clear_ that there is incredible appetite for tools that help us design reliable concurrent systems given the wild success of things like k8s, Kafka, AWS's distributed systems products, etc., but why hasn't Erlang/Elixir been able to capture that share? My friends and I debate this all the time, but I don't know the answer.
- bad_haircut72 2y ago
  Decision making in computing is mostly a matter of fashion / cargo-culting.
  - matthew16550 2y ago
    And Resume padding
- ngrilly 2y ago
  I think that’s mostly due to Erlang looking too alien compared to mainstream languages. Elixir is changing that but it arrived a bit late.
- jerf 2y ago
  "but why hasn't Erlang/Elixir been able to capture that share?" Because Erlang has a well-integrated collection of what are by 2025 standards mediocre tools. There is value to that integration, and I absolutely won't deny that. However, the state of the art has moved beyond Erlang in a number of ways, and you're taking a pretty big penalty to stick to BEAM on a number of fronts now. Its performance is sub-par, and if you're running a large cluster, that's actually going to matter. Erlang qua Erlang I'd call a subpar language, and Elixir qua Elixir is merely competitive; there are many places to get similar capabilities, with a wide variety of other available cost/benefit choices. Erlang's message bus is not terribly resilient itself; modern message busses can be resilient against individual nodes in the message bus going down, and it's a powerful pattern to have multiple consumers against a single queue, which Erlang's focus on PIDs tends to inhibit. Erlang's message bus is 0-or-1 when as near as I can tell the rest of the world has decided, correctly IMHO, that 1-or-n is superior. Erlang is fairly insular; once you have to hook up one non-BEAM service to the system, well, you're going to do that over some sort of message bus or something, and you pretty quickly get to the point that you might as well let that be your core architecture rather than the BEAM cluster. Once you're heterogeneous, and BEAM is just another node on the net, there isn't necessarily a lot of reason to stay there. And as a system scales up, the pull to heterogeneity approaches infinity; takes a lot of work to go to an entire company and force them to work entirely in BEAM. Plus, some of the problems Erlang solved in one way have developed better solutions. Erlang solves the problem of multiple code bases possibly simultaneously existing in the same cluster by basically making everything untyped. That was a nifty solution for the 1990s, but today I think we've gotten a lot better at having typed data structures that still retain backwards compatibility if necessary. So throwing away the entire type system, including all the methods and inheritance or composition or whatever, to solve that problem is a heck of a blow. I do want to close out with a repetition of the fact that there is value in that solid integration. More people today are aware of the various tools like "message busses", but it is still clearly not as common knowledge as I'd like and I still see entire teams struggling along basically crafting an ad-hoc half-specified custom message bus every so often, which in 2025 is insane. (I have written a couple of services where I have basically had to provide HTTP "REST" endpoints that end up just being proxies on to my internal message bus that my system is really based on, because they'd rather POST HTTP than have to use a message bus library, even though it doesn't really buy them anything.) Erlang does help educate people about what are now the basics of cloud architecture. And that "well-integrated collection of mediocre tools" can still solve a lot of problems. Many sins can be forgiven by a 32 4GHz cores backed by high powered RAM, disk, and networking. But it would take a lot of backwards-incompatible changes to create a BEAM 2.0 that would be competitive on all fronts... if indeed such a thing is even possible. The variety of techs exist for a reason. It stinks to have to paw through them sometimes, but the upside is you'll often find the exact right solution for your needs.
klabb3 2y ago
Go is my favorite language but: > Go doesn't need a "handle_event/2" because it has channels, and you should use those Of what type? But most importantly, channels are local to the process, so you need glue to make it networked. (I assume erlang has networked message handling abstracted away). In addition I’ve seen 3-4 different variations of your proposed pattern for long-running server like things. I agree fully that porting should make use of idiomatic constructs. But I also think languages can have hidden mechanics that loses the valuable essence while porting – a form of anti-relativism of PLs if you will. It’s entirely possible to me that this ”oh a channel? just wrap it in X” is much more detrimental to interop than what it sounds like. For instance take http.Handler in Go. Similarly simple but what are the real world implications of having it in std? An ecosystem of middleware that is largely compatible with one another, without pre-coordination (a non-std http server X can be used with auth middleware Y and logging middleware Z). Similar things can be said about io.Reader and friends. These extremely simply interfaces are arguably more valuable than the implementations. If, and I’m speculating here, Erlang got many of the interfaces for reliable distributed systems right, that can be what enables the whole.
- jerf 2y ago
  "Of what type?" Of the type of the messages you're sending. Which can either be an interface for multiple messages, or you can use multiple channels with one type each. I've done both. This is not an important question when actually programming in Go. "But most importantly, channels are local to the process, so you need glue to make it networked." This is an important consideration if you are using Go. Although I would observe that it isn't so much that "channels don't do network" as that "channels are a local tool"; e.g., we do not complain that OS mutexes are not "network capable", because they're intrinsically local. Network locking uses different solutions, and we don't really consider etcd a competitor to a local "lock" call. But there are dozens of message busses in the world now, and Erlang's isn't really all that competitive modulo its integration.

by jerf 2y ago

asa400 2y ago

To follow on from your excellent post, I think a reasonable next question is, "why have these kinds of approaches and ideas in other languages and systems succeeded in gaining market adoption, but Erlang/Elixir has not?" This to me is the most interesting question about Erlang, and I say this as someone who works professionally in Elixir. It's _clear_ that there is incredible appetite for tools that help us design reliable concurrent systems given the wild success of things like k8s, Kafka, AWS's distributed systems products, etc., but why hasn't Erlang/Elixir been able to capture that share? My friends and I debate this all the time, but I don't know the answer.

bad_haircut72 2y ago
Decision making in computing is mostly a matter of fashion / cargo-culting.
- matthew16550 2y ago
  And Resume padding
ngrilly 2y ago
I think that’s mostly due to Erlang looking too alien compared to mainstream languages. Elixir is changing that but it arrived a bit late.
jerf 2y ago
"but why hasn't Erlang/Elixir been able to capture that share?" Because Erlang has a well-integrated collection of what are by 2025 standards mediocre tools. There is value to that integration, and I absolutely won't deny that. However, the state of the art has moved beyond Erlang in a number of ways, and you're taking a pretty big penalty to stick to BEAM on a number of fronts now. Its performance is sub-par, and if you're running a large cluster, that's actually going to matter. Erlang qua Erlang I'd call a subpar language, and Elixir qua Elixir is merely competitive; there are many places to get similar capabilities, with a wide variety of other available cost/benefit choices. Erlang's message bus is not terribly resilient itself; modern message busses can be resilient against individual nodes in the message bus going down, and it's a powerful pattern to have multiple consumers against a single queue, which Erlang's focus on PIDs tends to inhibit. Erlang's message bus is 0-or-1 when as near as I can tell the rest of the world has decided, correctly IMHO, that 1-or-n is superior. Erlang is fairly insular; once you have to hook up one non-BEAM service to the system, well, you're going to do that over some sort of message bus or something, and you pretty quickly get to the point that you might as well let that be your core architecture rather than the BEAM cluster. Once you're heterogeneous, and BEAM is just another node on the net, there isn't necessarily a lot of reason to stay there. And as a system scales up, the pull to heterogeneity approaches infinity; takes a lot of work to go to an entire company and force them to work entirely in BEAM. Plus, some of the problems Erlang solved in one way have developed better solutions. Erlang solves the problem of multiple code bases possibly simultaneously existing in the same cluster by basically making everything untyped. That was a nifty solution for the 1990s, but today I think we've gotten a lot better at having typed data structures that still retain backwards compatibility if necessary. So throwing away the entire type system, including all the methods and inheritance or composition or whatever, to solve that problem is a heck of a blow. I do want to close out with a repetition of the fact that there is value in that solid integration. More people today are aware of the various tools like "message busses", but it is still clearly not as common knowledge as I'd like and I still see entire teams struggling along basically crafting an ad-hoc half-specified custom message bus every so often, which in 2025 is insane. (I have written a couple of services where I have basically had to provide HTTP "REST" endpoints that end up just being proxies on to my internal message bus that my system is really based on, because they'd rather POST HTTP than have to use a message bus library, even though it doesn't really buy them anything.) Erlang does help educate people about what are now the basics of cloud architecture. And that "well-integrated collection of mediocre tools" can still solve a lot of problems. Many sins can be forgiven by a 32 4GHz cores backed by high powered RAM, disk, and networking. But it would take a lot of backwards-incompatible changes to create a BEAM 2.0 that would be competitive on all fronts... if indeed such a thing is even possible. The variety of techs exist for a reason. It stinks to have to paw through them sometimes, but the upside is you'll often find the exact right solution for your needs.

klabb3 2y ago

Go is my favorite language but: > Go doesn't need a "handle_event/2" because it has channels, and you should use those Of what type? But most importantly, channels are local to the process, so you need glue to make it networked. (I assume erlang has networked message handling abstracted away). In addition I’ve seen 3-4 different variations of your proposed pattern for long-running server like things. I agree fully that porting should make use of idiomatic constructs. But I also think languages can have hidden mechanics that loses the valuable essence while porting – a form of anti-relativism of PLs if you will. It’s entirely possible to me that this ”oh a channel? just wrap it in X” is much more detrimental to interop than what it sounds like. For instance take http.Handler in Go. Similarly simple but what are the real world implications of having it in std? An ecosystem of middleware that is largely compatible with one another, without pre-coordination (a non-std http server X can be used with auth middleware Y and logging middleware Z). Similar things can be said about io.Reader and friends. These extremely simply interfaces are arguably more valuable than the implementations. If, and I’m speculating here, Erlang got many of the interfaces for reliable distributed systems right, that can be what enables the whole.

jerf 2y ago
"Of what type?" Of the type of the messages you're sending. Which can either be an interface for multiple messages, or you can use multiple channels with one type each. I've done both. This is not an important question when actually programming in Go. "But most importantly, channels are local to the process, so you need glue to make it networked." This is an important consideration if you are using Go. Although I would observe that it isn't so much that "channels don't do network" as that "channels are a local tool"; e.g., we do not complain that OS mutexes are not "network capable", because they're intrinsically local. Network locking uses different solutions, and we don't really consider etcd a competitor to a local "lock" call. But there are dozens of message busses in the world now, and Erlang's isn't really all that competitive modulo its integration.