4 ms·

> if raw computational performance is high on your list of needs it's got some nontrivial disadvantages. That's what Rustler, Zigler, and Numerical Elixir (Nx)

> if raw computational performance is high on your list of needs it's got some nontrivial disadvantages. That's what Rustler, Zigler, and Numerical Elixir (Nx) solve. It doesn't mitigate issues around deep copies when passing messages between processes, but the absence of shared memory means that race conditions aren't a problem to worry about, and the memory model is such that garbage collection is never going to be a performance issue even if you scale up to millions of processes per node. It also means you can trivially scale horizontally across multiple nodes because your code doesn't (more accurately can't) depend on sharing memory.

jerf 3y ago
I've been hearing "You can use this slow language and just plug in to a faster one when you need more speed" for over two decades now, and it still has yet to impress me in practice. It's still too easy to accidentally stray back into the slow language for extended periods, and it's virtually guaranteed when it's the "glue layer". Rust straight-up solves race conditions on its own terms, it is not hard to program Go in a way that doesn't have them, etc. It's easier just to write in the fast language in the first place. Also, I think the BEAM GC is widely misunderstood. Per process it's kinda neat that it can GC without having to consider the others. But globally it's actually kinda slow. Like I said, I've straight-up ported Erlang programs into Go, network clustering and all, and Go was quite significantly faster, even with the not-so-great GC Go had in the ~1.6 era. (I suspect if I ran the same code now it would just work with no GC problems.) BEAM GC is more "hey, look at this neat trick!" than "better than anything around". I actually wouldn't consider it very good at all in modern terms. The Erlang community has a huge problem with thinking it's still 2003 and comparing Erlang to all the other competition in 2003, but unless you're still living in 2003, you need to be comparing to what's around in 2023.
- h0l0cube 3y ago
  > It's easier just to write in the fast language in the first place. I’m not sure if that’s measurably true in terms of ergonomics (e.g., there’s no cognitive load difference between using a borrow checker, types, programming defensively, thinking about async at all). But I have no actual experience with go to offer a compelling counterpoint > Like I said, I've straight-up ported Erlang programs into Go, network clustering and all, and Go was quite significantly faster Was the JIT any help? In your view, what technical decisions do you think prevent performance in Erlang VM? Why do you think good performance either impossible, difficult to achieve, or unlikely due to these reasons?
  - jerf 3y ago
    "I’m not sure if that’s measurably true in terms of ergonomics (e.g., there’s no cognitive load difference between using a borrow checker, types, programming defensively, thinking about async at all). But I have no actual experience with go to offer a compelling counterpoint" The differentiation used to be HUGE. In the 200xs, especially earlier in that era, static languages were a NIGHTMARE. I personally believe this is a large part of the reason that the dynamic scripting languages won as much of the market as they did, despite their manifold disadvantages. They were going up against language environments that were even worse, by a large margin. 1990s C++ was just a nightmare. Today the difference is much more muted. I've had the pleasure of being assigned a greenfield research project by myself for the last 6 weeks, and I'm not only making plenty of progress in such a task in a statically typed language (Go), I am now well into the point where I am actually better off in a static language, because I can make very significant infrastructure changes safely, without having to bodge adaptors everywhere. But it's still an opinion thing, sure. "performance issues" The super weak typing story is a big problem. We've seen a whole bunch of attempts over the years to take a weakly typed language and speed it up with JITs. In a nutshell, the answer is, you can do better, but we didn't get anywhere near C speed performance in general like there was so much hope in the early 2000s... and the programming community loves imbibing old propaganda and still has not really grappled with the fact that JITs do not generally "work" in that sense. Linked lists as a primary data structure for so many things is not good in 2023. While you avoid some of their issues in that the other end of the linked list is much more likely to be in cache because of the smallness of the local OS process, they're still linked lists. The BEAM VM does a lot of bookkeeping. When this bookkeeping is useful, it is invaluable, but when you have a system just chugging along, working, and not needing any intervention, it's doing a lot of unnecessary bookkeeping that most other languages aren't. Note I'm not saying this is all bad; I'm just saying, it isn't free. Pattern matching is probably generally slower than polymorphic dispatch, let alone static type-based dispatch. Especially when you start writing deep patterns. I'm a little heterodox on this question in general, probably. I'm actually not entirely convinced that pattern matching is somehow generally "superior" than the alternatives. There are several negatives I'd associate with it that advocates often overlook. It's not disastrously broken or anything, but I find its cost/benefit over alternatives to much more complicated than often supposed. Immutability slows down a language. Compilers can try to optimize around it in the common case, but while it may have software engineering disadvantages it is certainly generally faster. While it's easy to show immutability has a maximum "log n" slowdown factor over mutable code, it's still easy to accidentally do worse than that, and a "log n" factor is still something you'll feel. It's a lot of little things. And like I said, the totality still adds up to a usable language. It's still generally faster than Python, for what that's worth. But it's also not a fast language. The BEAM VM itself performs well but the language it implements is slow, and I think in the intervening years there's been a lot of languages taking a lot of different approaches that achieves the same goals without the same slowdowns.

jerf 3y ago

I've been hearing "You can use this slow language and just plug in to a faster one when you need more speed" for over two decades now, and it still has yet to impress me in practice. It's still too easy to accidentally stray back into the slow language for extended periods, and it's virtually guaranteed when it's the "glue layer". Rust straight-up solves race conditions on its own terms, it is not hard to program Go in a way that doesn't have them, etc. It's easier just to write in the fast language in the first place. Also, I think the BEAM GC is widely misunderstood. Per process it's kinda neat that it can GC without having to consider the others. But globally it's actually kinda slow. Like I said, I've straight-up ported Erlang programs into Go, network clustering and all, and Go was quite significantly faster, even with the not-so-great GC Go had in the ~1.6 era. (I suspect if I ran the same code now it would just work with no GC problems.) BEAM GC is more "hey, look at this neat trick!" than "better than anything around". I actually wouldn't consider it very good at all in modern terms. The Erlang community has a huge problem with thinking it's still 2003 and comparing Erlang to all the other competition in 2003, but unless you're still living in 2003, you need to be comparing to what's around in 2023.

h0l0cube 3y ago
> It's easier just to write in the fast language in the first place. I’m not sure if that’s measurably true in terms of ergonomics (e.g., there’s no cognitive load difference between using a borrow checker, types, programming defensively, thinking about async at all). But I have no actual experience with go to offer a compelling counterpoint > Like I said, I've straight-up ported Erlang programs into Go, network clustering and all, and Go was quite significantly faster Was the JIT any help? In your view, what technical decisions do you think prevent performance in Erlang VM? Why do you think good performance either impossible, difficult to achieve, or unlikely due to these reasons?
- jerf 3y ago
  "I’m not sure if that’s measurably true in terms of ergonomics (e.g., there’s no cognitive load difference between using a borrow checker, types, programming defensively, thinking about async at all). But I have no actual experience with go to offer a compelling counterpoint" The differentiation used to be HUGE. In the 200xs, especially earlier in that era, static languages were a NIGHTMARE. I personally believe this is a large part of the reason that the dynamic scripting languages won as much of the market as they did, despite their manifold disadvantages. They were going up against language environments that were even worse, by a large margin. 1990s C++ was just a nightmare. Today the difference is much more muted. I've had the pleasure of being assigned a greenfield research project by myself for the last 6 weeks, and I'm not only making plenty of progress in such a task in a statically typed language (Go), I am now well into the point where I am actually better off in a static language, because I can make very significant infrastructure changes safely, without having to bodge adaptors everywhere. But it's still an opinion thing, sure. "performance issues" The super weak typing story is a big problem. We've seen a whole bunch of attempts over the years to take a weakly typed language and speed it up with JITs. In a nutshell, the answer is, you can do better, but we didn't get anywhere near C speed performance in general like there was so much hope in the early 2000s... and the programming community loves imbibing old propaganda and still has not really grappled with the fact that JITs do not generally "work" in that sense. Linked lists as a primary data structure for so many things is not good in 2023. While you avoid some of their issues in that the other end of the linked list is much more likely to be in cache because of the smallness of the local OS process, they're still linked lists. The BEAM VM does a lot of bookkeeping. When this bookkeeping is useful, it is invaluable, but when you have a system just chugging along, working, and not needing any intervention, it's doing a lot of unnecessary bookkeeping that most other languages aren't. Note I'm not saying this is all bad; I'm just saying, it isn't free. Pattern matching is probably generally slower than polymorphic dispatch, let alone static type-based dispatch. Especially when you start writing deep patterns. I'm a little heterodox on this question in general, probably. I'm actually not entirely convinced that pattern matching is somehow generally "superior" than the alternatives. There are several negatives I'd associate with it that advocates often overlook. It's not disastrously broken or anything, but I find its cost/benefit over alternatives to much more complicated than often supposed. Immutability slows down a language. Compilers can try to optimize around it in the common case, but while it may have software engineering disadvantages it is certainly generally faster. While it's easy to show immutability has a maximum "log n" slowdown factor over mutable code, it's still easy to accidentally do worse than that, and a "log n" factor is still something you'll feel. It's a lot of little things. And like I said, the totality still adds up to a usable language. It's still generally faster than Python, for what that's worth. But it's also not a fast language. The BEAM VM itself performs well but the language it implements is slow, and I think in the intervening years there's been a lot of languages taking a lot of different approaches that achieves the same goals without the same slowdowns.