13 ms·

Optimising for Concurrency: Comparing the BEAM and JVM virtual machines

185 points by francescoc 6y ago

hinkley 6y ago
Can someone steer me to some good benchmarks, discussions of perf characteristics and gotchas of the BEAM? My search-fu is weak and I'm not finding the sort of content I'm after. I'm trying to learn Elixir and being a systems thinker so before I (can) get too comfortable I'm gonna want to dive into origin stories to build up my holistic map of why things are the way they are, what can be done and what can't be done, and understanding bottlenecks in the BEAM seems like it's gonna have to be part of that (the way I studied JVM tech documentation when I did perf and architecture work in Java)
- toast0 6y ago
  I think you're going to have a hard time finding what you're after. Erlang in Anger [1] might be the closest, it will at least show some of the gotchas you run into. From my experience, the gotchas tend to hit with emergent behavior, which is hard to benchmark, and may be repeatable in production, but is hard to model in a testing framework. I'm not sure how much impact off-heap messaging has had, but the basic gotcha is that as a process gets bigger, it tends to run slower (because GC over more memory takes longer), and develop a larger message queue, which makes it slower. You need to have backpressure in your system, or small blips in procesing can blow up to huge messaging queues that can't be processed. Monitoring for overall queue size and maximum queue size is an important health indicator. The other basic gotcha is that Erlang/OTP tends to default to 'unlimited' resource limits and 'infinity' time outs. You often want to have limits, and timeouts, but a general system doesn't know what you want. Sometimes, the unlimited settings result in terrible system behavior if you hit larger numbers than anyone else tested, but if you hit this, it's usually easy to fix. A good thing about OTP is that they've written as much as possible of the environment in Erlang itself, so it's easier to change things when needed than a system where most of the provided apis are implemented in C. [1] https://erlang-in-anger.com/ https://erlang-in-anger.com/
- micmus 6y ago
  In general I would say there's no good single book or resource that describes everything comprehensively. There's a lot of resources, though, but mostly scattered in various places. The BEAM Book [1] is a good, though unfinished resource talking in general about the implementation - the memory model and the interpreter. If you're interested in some very low-level details of the runtime, the internal documentation [2] also holds a lot of interesting details. There are also some additional details on internals at Spawned Shelter [3]. [1]: https://blog.stenmans.org/theBeamBook/ https://blog.stenmans.org/theBeamBook/ [2]: https://github.com/erlang/otp/tree/master/erts/emulator/internal_doc https://github.com/erlang/otp/tree/master/erts/emulator/inte... [3]: http://spawnedshelter.com/#erlang-design-choices-and-beam-internals http://spawnedshelter.com/#erlang-design-choices-and-beam-in...
  - hinkley 6y ago
    I want to know the constraints to, and evolution of, sequential computation on the BEAM. I want to form opinions on how that landscape is likely to change within the lifespan of a project I'm affiliated with. I get mostly false positives trying to find those sorts of discussions or metrics.
    - di4na 6y ago
      I am not sure i understand the problem you are trying to find information about. Maybe explain it a little bit more ? or go ask for it in the elixir forum, people can try to be your librarians there
      - hinkley 6y ago
        To an outsider, it seems like the BEAM documentation [and particularly, videos] go out of their way to discuss how process management and IPC communication works and how certain classes of data are managed. They talk about what makes the BEAM the BEAM to exclusion of all other concerns. Prior to finding this document (http://www.cs-lab.org/historical_beam_instruction_set.html http://www.cs-lab.org/historical_beam_instruction_set.html) I had no idea whether you could actually do computation on the BEAM. I was starting to wonder if they had misappropriated the term VM, and some sort of inline assembly trick was being used for everything but control flow and IPC. Interpreted code has very, very real computational constraints and you can't assume people will know this, even now. Especially if your system is noteworthy for how it is not like other systems. Where does it stop being 'weird' and start being conventional? The boundaries describe both sides of a distinction. Even if you're only interested in the exotic part, leave some breadcrumbs for others.
jph 6y ago
BEAM is amazing and IMHO there's one very sweet spot ready for optimization: math functions. I know I can escape out to C/Rust/etc. yet the majority of what I do is simple float math such as stddev and vector normalization. The article states benchmark of 5000% speedup on floats when switching from BEAM to the JVM. I would like to offer $100 as a gift incentive to anyone here who wants to work on optimizing BEAM math.
- chrisseaton 6y ago
  People say this isn't what BEAM is intended for an it excels elsewhere, which yes I'm sure it does. But why can't it be both? Why can't you do everything that BEAM does... and then also have an optimising JIT for the straight line maths code? Couldn't you leave all the other parts of the system the same and keep all the existing benefits? Improving one doesn't damage the other does it?
  - olikas 6y ago
    Co-author here. The problem with number crunching or maths is that it is very difficult to cut the whole computation into smaller units and pre-emptively schedule it. If it is possible for a specific use case, then it is moderately easy to replace that part with NIFs. For effective maths you need to convert the internal tagged number representation to machine native code that is also expensive. Solving these two things in the generic case is very difficult while preserving all the good parts.
    - PopeDotNinja 6y ago
      Am I correct in saying that functions written in C do not get pre-empted like Erlang functions? If that is true, you could write computationally intense code in C within a BEAM app. But I think this misses the point. Pre-emption is really cool for concurrency abstractions, and the trade off is being less good at single threaded computation. Trying to turn Erlang into something like a Bitcoin miner is kind of like combining a bunch of Roombas to make a Shop-Vac.
Traubenfuchs 6y ago
> Programming with concurrency primitives is a difficult task because of the challenges created by its shared memory model. I never understood this often repeated point. As junior / mid-level developer I had the privilege to run self written .jar files on government scale systems with more than 50 cores. I used Java thread pools and concurrent data structures to do heavy cross thread caching. It was all pretty simple and concurrency & parallelism were never an issue but simply a necessity to make things run fast enough. Am I a concurrent programming genius? Were the types of problems/challenges I was solving too simple? When is concurrency in Java ever hard+? + I know about Java masterpieces like the LMAX Disruptor that are mostly beyond my skill level, but those are low level writte-once libraries you wouldn't write yourself.
- vlovich123 6y ago
  It's fine if you know what you're doing and are the primary maintainer. As someone who's encountered code maintained over a long period of time with lots of people coming and going, concurrency using locks is quite ugly, especially as people cargo cult "better performing" solutions that aren't actually & just add complexity/race conditions. My experience is primarily C/C++ but this is all agnostic to the language.
  - deleted 6y ago
    [deleted]
  - trhway 6y ago
    > this is all agnostic to the language. yes and no. Yes - in the sense that pretty equivalent things can be done in different languages. No - i'm in the same group of "geniuses" as GP, and i see for example on our current huge C++ platform project the highly technical people struggle with and do the wrong things with concurrency/multithreading that i don't remember seeing the even mildly technical people doing on various large Java projects.
- justinrstout 6y ago
  "Concurrency primitives" here is probably referring to Java's fundamental mutex system as used with `synchronized`, `wait()`, and `notify()`. Java's thread pools and concurrent data structures are built on top of these and as you noted are relatively straightforward to use correctly, as they take care of the actual coordination of threads for you.
shawnz 6y ago
The JVM supports hot code loading, although this article seems to imply only BEAM supports it.
- brightball 6y ago
  Do you have a reference on that? I want to make sure you're talking about the same thing.
  - orestis 6y ago
    Clojure does hot code reloading as a built in. You essentially send code to a running system and you change it. It’s enabled by a dynamic class loader. I wouldn’t say it’s common outside of Clojure though, the whole language and ecosystem is built around this concept. To be clear: JVM enables the feature, so “technically” JVM allows hot code reload. Not sure how useful this is in practice for non-Clojure JVM users.
    - shawnz 6y ago
      Eclipse, IntelliJ and Netbeans all support it out of the box for Java code.
    - eggsnbacon1 6y ago
      Runtime code generation is a common optimization in java frameworks. End-users may never see it but the majority of popular frameworks use it under the covers. Debuggers also use the functionality to allow live code editing and expression evaluation when paused on a breakpoint
  - gmfawcett 6y ago
    ClassLoader [1] in the small, and OSGi [2] in the large, are good starting points for comparison. [1] https://docs.oracle.com/javase/7/docs/api/java/lang/ClassLoader.html https://docs.oracle.com/javase/7/docs/api/java/lang/ClassLoa... [2] https://en.wikipedia.org/wiki/OSGi https://en.wikipedia.org/wiki/OSGi
  - shawnz 6y ago
ForHackernews 6y ago
Apparently this is an Erlang BEAM, not Apache BEAM https://beam.apache.org/ https://beam.apache.org/?
- pdimitar 6y ago
  Yes. It's about Erlang's BEAM VM.
eggsnbacon1 6y ago
the author leaves out Kotlin which adds support for coroutines on the language level and still compiles to java bytecode. These are not classic continuations because they cannot be cancelled, but they're still very useful and true fibers. There's also the Quasar library that adds fiber support to existing Java projects, but its mostly unmaintained since the maintainers were pulled in to work on Project Loom. Then there's Project Loom, an active branch of of OpenJDK with language support for continuations and a fiber threading model. The prototype is done and they're in the optimization phase. I expect fibers to land in the Java spec somewhere around JDK 17. I figure its fair to mention these as the authors criticisms are somewhat valid but will not be for very long (few years max?) In summary: Java will have true fiber support "soon". This will invalidate the arguments for Erlang concurrency model. They are already outdated if you are okay using mixed java/kotlin coroutines or Quasar library The newer Java GC's Shenandoah and ZGC address authors criticisms of pause times. They already exist, are free, and are in stable releases. Dare I say they are almost certainly better than Erlang's GC. They are truly state of the art, arguably far superior to the GC's used in Go, .NET, etc. Pause times are ~10 milliseconds at 99.5%ile latency for multi terabyte heaps, with average pause times well below 1 millisecond. No other GC'ed language comes close to my knowledge. His points 1 and 2 no longer exist with these collectors. You don't need 2X memory for the copy phase and the collectors quickly return unused memory to the OS. This has been the case for several years. Hot code reloading. JVM supports this extensively and its used all the time. Look into ByteBuddy, CGLIB, ASM, Spring AOP if you want to know more. Java also supports code generation at build time using Annotation Processors. This is also extensively used/abused to get rid of language cruft
- throw51319 6y ago
  Is Java the best out there?
  - The_rationalist 6y ago
    Kotlin* but yes JVM is basically the best platform
  - deleted 6y ago
    [deleted]
deleted 6y ago
[deleted]
alfanerd 6y ago
Of interest might also be Erjang, Kresten Krab's port of BEAM to JVM. https://github.com/trifork/erjang https://github.com/trifork/erjang As I understand it, it is feature complete and actually runs Erlang pretty well. Could be interesting to see some benchmark testing.
exabrial 6y ago
They mentioned this for the beam virtual machine but not for the JVM, the JVM actually can also do hot code loading as long as call site signatures are not changed or added. In some cases you you can make major changes to the current stack and restart the frame which is a pretty handy feature for developer. Some commercial extensions to the JVM get around all of these limitations.
jeffrallen 6y ago
Tl,dr. The point seems to be, "shared nothing makes concurrency and GC easy". Congrats. But also lots of big fast systems use shared memory, so just relax, STFU, and understand that tradeoffs exist.

185 points by francescoc 6y ago

Can someone steer me to some good benchmarks, discussions of perf characteristics and gotchas of the BEAM? My search-fu is weak and I'm not finding the sort of content I'm after. I'm trying to learn Elixir and being a systems thinker so before I (can) get too comfortable I'm gonna want to dive into origin stories to build up my holistic map of why things are the way they are, what can be done and what can't be done, and understanding bottlenecks in the BEAM seems like it's gonna have to be part of that (the way I studied JVM tech documentation when I did perf and architecture work in Java)

toast0 6y ago
I think you're going to have a hard time finding what you're after. Erlang in Anger [1] might be the closest, it will at least show some of the gotchas you run into. From my experience, the gotchas tend to hit with emergent behavior, which is hard to benchmark, and may be repeatable in production, but is hard to model in a testing framework. I'm not sure how much impact off-heap messaging has had, but the basic gotcha is that as a process gets bigger, it tends to run slower (because GC over more memory takes longer), and develop a larger message queue, which makes it slower. You need to have backpressure in your system, or small blips in procesing can blow up to huge messaging queues that can't be processed. Monitoring for overall queue size and maximum queue size is an important health indicator. The other basic gotcha is that Erlang/OTP tends to default to 'unlimited' resource limits and 'infinity' time outs. You often want to have limits, and timeouts, but a general system doesn't know what you want. Sometimes, the unlimited settings result in terrible system behavior if you hit larger numbers than anyone else tested, but if you hit this, it's usually easy to fix. A good thing about OTP is that they've written as much as possible of the environment in Erlang itself, so it's easier to change things when needed than a system where most of the provided apis are implemented in C. [1] https://erlang-in-anger.com/ https://erlang-in-anger.com/
micmus 6y ago
In general I would say there's no good single book or resource that describes everything comprehensively. There's a lot of resources, though, but mostly scattered in various places. The BEAM Book [1] is a good, though unfinished resource talking in general about the implementation - the memory model and the interpreter. If you're interested in some very low-level details of the runtime, the internal documentation [2] also holds a lot of interesting details. There are also some additional details on internals at Spawned Shelter [3]. [1]: https://blog.stenmans.org/theBeamBook/ https://blog.stenmans.org/theBeamBook/ [2]: https://github.com/erlang/otp/tree/master/erts/emulator/internal_doc https://github.com/erlang/otp/tree/master/erts/emulator/inte... [3]: http://spawnedshelter.com/#erlang-design-choices-and-beam-internals http://spawnedshelter.com/#erlang-design-choices-and-beam-in...
- hinkley 6y ago
  I want to know the constraints to, and evolution of, sequential computation on the BEAM. I want to form opinions on how that landscape is likely to change within the lifespan of a project I'm affiliated with. I get mostly false positives trying to find those sorts of discussions or metrics.
  - di4na 6y ago
    I am not sure i understand the problem you are trying to find information about. Maybe explain it a little bit more ? or go ask for it in the elixir forum, people can try to be your librarians there
    - hinkley 6y ago
      To an outsider, it seems like the BEAM documentation [and particularly, videos] go out of their way to discuss how process management and IPC communication works and how certain classes of data are managed. They talk about what makes the BEAM the BEAM to exclusion of all other concerns. Prior to finding this document (http://www.cs-lab.org/historical_beam_instruction_set.html http://www.cs-lab.org/historical_beam_instruction_set.html) I had no idea whether you could actually do computation on the BEAM. I was starting to wonder if they had misappropriated the term VM, and some sort of inline assembly trick was being used for everything but control flow and IPC. Interpreted code has very, very real computational constraints and you can't assume people will know this, even now. Especially if your system is noteworthy for how it is not like other systems. Where does it stop being 'weird' and start being conventional? The boundaries describe both sides of a distinction. Even if you're only interested in the exotic part, leave some breadcrumbs for others.

jph 6y ago

BEAM is amazing and IMHO there's one very sweet spot ready for optimization: math functions. I know I can escape out to C/Rust/etc. yet the majority of what I do is simple float math such as stddev and vector normalization. The article states benchmark of 5000% speedup on floats when switching from BEAM to the JVM. I would like to offer $100 as a gift incentive to anyone here who wants to work on optimizing BEAM math.

chrisseaton 6y ago
People say this isn't what BEAM is intended for an it excels elsewhere, which yes I'm sure it does. But why can't it be both? Why can't you do everything that BEAM does... and then also have an optimising JIT for the straight line maths code? Couldn't you leave all the other parts of the system the same and keep all the existing benefits? Improving one doesn't damage the other does it?
- olikas 6y ago
  Co-author here. The problem with number crunching or maths is that it is very difficult to cut the whole computation into smaller units and pre-emptively schedule it. If it is possible for a specific use case, then it is moderately easy to replace that part with NIFs. For effective maths you need to convert the internal tagged number representation to machine native code that is also expensive. Solving these two things in the generic case is very difficult while preserving all the good parts.
  - PopeDotNinja 6y ago
    Am I correct in saying that functions written in C do not get pre-empted like Erlang functions? If that is true, you could write computationally intense code in C within a BEAM app. But I think this misses the point. Pre-emption is really cool for concurrency abstractions, and the trade off is being less good at single threaded computation. Trying to turn Erlang into something like a Bitcoin miner is kind of like combining a bunch of Roombas to make a Shop-Vac.

Traubenfuchs 6y ago

> Programming with concurrency primitives is a difficult task because of the challenges created by its shared memory model. I never understood this often repeated point. As junior / mid-level developer I had the privilege to run self written .jar files on government scale systems with more than 50 cores. I used Java thread pools and concurrent data structures to do heavy cross thread caching. It was all pretty simple and concurrency & parallelism were never an issue but simply a necessity to make things run fast enough. Am I a concurrent programming genius? Were the types of problems/challenges I was solving too simple? When is concurrency in Java ever hard+? + I know about Java masterpieces like the LMAX Disruptor that are mostly beyond my skill level, but those are low level writte-once libraries you wouldn't write yourself.

vlovich123 6y ago
It's fine if you know what you're doing and are the primary maintainer. As someone who's encountered code maintained over a long period of time with lots of people coming and going, concurrency using locks is quite ugly, especially as people cargo cult "better performing" solutions that aren't actually & just add complexity/race conditions. My experience is primarily C/C++ but this is all agnostic to the language.
- deleted 6y ago
  [deleted]
- trhway 6y ago
  > this is all agnostic to the language. yes and no. Yes - in the sense that pretty equivalent things can be done in different languages. No - i'm in the same group of "geniuses" as GP, and i see for example on our current huge C++ platform project the highly technical people struggle with and do the wrong things with concurrency/multithreading that i don't remember seeing the even mildly technical people doing on various large Java projects.
justinrstout 6y ago
"Concurrency primitives" here is probably referring to Java's fundamental mutex system as used with `synchronized`, `wait()`, and `notify()`. Java's thread pools and concurrent data structures are built on top of these and as you noted are relatively straightforward to use correctly, as they take care of the actual coordination of threads for you.

shawnz 6y ago

The JVM supports hot code loading, although this article seems to imply only BEAM supports it.

brightball 6y ago
Do you have a reference on that? I want to make sure you're talking about the same thing.
- orestis 6y ago
  Clojure does hot code reloading as a built in. You essentially send code to a running system and you change it. It’s enabled by a dynamic class loader. I wouldn’t say it’s common outside of Clojure though, the whole language and ecosystem is built around this concept. To be clear: JVM enables the feature, so “technically” JVM allows hot code reload. Not sure how useful this is in practice for non-Clojure JVM users.
  - shawnz 6y ago
    Eclipse, IntelliJ and Netbeans all support it out of the box for Java code.
  - eggsnbacon1 6y ago
    Runtime code generation is a common optimization in java frameworks. End-users may never see it but the majority of popular frameworks use it under the covers. Debuggers also use the functionality to allow live code editing and expression evaluation when paused on a breakpoint
- gmfawcett 6y ago
  ClassLoader [1] in the small, and OSGi [2] in the large, are good starting points for comparison. [1] https://docs.oracle.com/javase/7/docs/api/java/lang/ClassLoader.html https://docs.oracle.com/javase/7/docs/api/java/lang/ClassLoa... [2] https://en.wikipedia.org/wiki/OSGi https://en.wikipedia.org/wiki/OSGi
- shawnz 6y ago

ForHackernews 6y ago

Apparently this is an Erlang BEAM, not Apache BEAM https://beam.apache.org/ https://beam.apache.org/?

pdimitar 6y ago
Yes. It's about Erlang's BEAM VM.

eggsnbacon1 6y ago

the author leaves out Kotlin which adds support for coroutines on the language level and still compiles to java bytecode. These are not classic continuations because they cannot be cancelled, but they're still very useful and true fibers. There's also the Quasar library that adds fiber support to existing Java projects, but its mostly unmaintained since the maintainers were pulled in to work on Project Loom. Then there's Project Loom, an active branch of of OpenJDK with language support for continuations and a fiber threading model. The prototype is done and they're in the optimization phase. I expect fibers to land in the Java spec somewhere around JDK 17. I figure its fair to mention these as the authors criticisms are somewhat valid but will not be for very long (few years max?) In summary: Java will have true fiber support "soon". This will invalidate the arguments for Erlang concurrency model. They are already outdated if you are okay using mixed java/kotlin coroutines or Quasar library The newer Java GC's Shenandoah and ZGC address authors criticisms of pause times. They already exist, are free, and are in stable releases. Dare I say they are almost certainly better than Erlang's GC. They are truly state of the art, arguably far superior to the GC's used in Go, .NET, etc. Pause times are ~10 milliseconds at 99.5%ile latency for multi terabyte heaps, with average pause times well below 1 millisecond. No other GC'ed language comes close to my knowledge. His points 1 and 2 no longer exist with these collectors. You don't need 2X memory for the copy phase and the collectors quickly return unused memory to the OS. This has been the case for several years. Hot code reloading. JVM supports this extensively and its used all the time. Look into ByteBuddy, CGLIB, ASM, Spring AOP if you want to know more. Java also supports code generation at build time using Annotation Processors. This is also extensively used/abused to get rid of language cruft

throw51319 6y ago
Is Java the best out there?
- The_rationalist 6y ago
  Kotlin* but yes JVM is basically the best platform
- deleted 6y ago
  [deleted]

deleted 6y ago

[deleted]

alfanerd 6y ago

Of interest might also be Erjang, Kresten Krab's port of BEAM to JVM. https://github.com/trifork/erjang https://github.com/trifork/erjang As I understand it, it is feature complete and actually runs Erlang pretty well. Could be interesting to see some benchmark testing.

exabrial 6y ago

They mentioned this for the beam virtual machine but not for the JVM, the JVM actually can also do hot code loading as long as call site signatures are not changed or added. In some cases you you can make major changes to the current stack and restart the frame which is a pretty handy feature for developer. Some commercial extensions to the JVM get around all of these limitations.

jeffrallen 6y ago

Tl,dr. The point seems to be, "shared nothing makes concurrency and GC easy". Congrats. But also lots of big fast systems use shared memory, so just relax, STFU, and understand that tradeoffs exist.