6 ms·

There is no doubt that JVM can provide massively better performance for a lot of stuff, but I wouldn't be suprised if Beam kicked JVM's butt in at least some ar

by patrec 7y ago

There is no doubt that JVM can provide massively better performance for a lot of stuff, but I wouldn't be suprised if Beam kicked JVM's butt in at least some areas. It's pretty much the only system I know of that provides for massive numbers of fairly scheduled, preemptive green threads. To do so it makes trade-offs that are not generally attractive, but I'd be surprised if it didn't beat the JVM here.

pron 7y ago
I would be. BEAM is just a very slow platform, not just compared to Java, but also compared to JavaScript and Go. Again, that's not to say it's not very useful for some things (although many Erlang apps are really mixed Erlang/C apps because of BEAM's performance). Hey, CPython is really slow, too and yet very useful. Green threads is another matter as it's not a feature that currently exists on the Java platform (there are somewhat similar features in Java platform languages like Kotlin's coroutines), but there is currently a project (that I'm leading) to add fibers (AKA user-mode threads, AKA lightweight threads, AKA green threads) to the JVM: https://wiki.openjdk.java.net/display/loom/ https://wiki.openjdk.java.net/display/loom/
- mruts 7y ago
  There’s Akka. Also Scala has Future’s. Both are very similar to green threads.
- chrisseaton 7y ago
  It's true that BEAM has excellent concurrency, parallelism, supervision, message passing... but I don't know understand why that has to come at the expense of being terrible at everything else. Why doesn't it have even a baseline reasonable quality JIT? Why doesn't its GC use a design more recent than half a century ago? It's got big corporate backers and PhDs working on it.
  - pron 7y ago
    I would imagine that's because they'd rather focus whatever resources they have on the needs and priorities of their target audience, which probably don't think performance is critical. I hope that one day TruffleBEAM+Loom would be a great platform for all those wonderful Erlang apps.
    - zambal 7y ago
      Searching for TruffleBEAM+Loom doesn't yield any meaningful results. Could you provide any info about what it is?
      - deleted 7y ago
        [deleted]
      - pron 7y ago
        It's something that I hope would exist one day. Loom is a project that I'm working on to add lightweight threads to the Java platform (https://wiki.openjdk.java.net/display/loom/ https://wiki.openjdk.java.net/display/loom/) and Truffle is a framework that allows writing fast language runtimes for the Java platform or standalone that chrisseaton is working on (https://github.com/oracle/graal/tree/master/truffle https://github.com/oracle/graal/tree/master/truffle).
  - jerf 7y ago
    "but I don't know understand why that has to come at the expense of being terrible at everything else." A great deal of it is certainly just resource poverty compared to the JVM or the various JS runtimes. It may have some nice high-powered resources dedicated to it, but I'm still very comfortable with pron's assessment that the JVM has literally 100x more resources poured into it. There are also some decisions that I would consider suboptimal for performance in the original Erlang specification though: 1. It extensively uses linked lists as a data structure. It was very fashionable in functional programming at the time, but the performance impact has in relative terms gotten worse as CPUs continue to speed up relative to RAM. Erlang does recover some of this vs. other functional programming languages in that the process model tends to keep the linked list components closer together together in RAM because they'll stay in the process' arena rather than being spread arbitrarily out over RAM, so walking a linked list a couple times is at least very likely to fit well into L1, but this is still going to be a pervasive loss of performance. 2. Haskell has done a lot of work in how to reconcile immutability with performance, and I'd still say there's a penalty there. Erlang hasn't, and a lot of it wouldn't apply (Erlang is strict), so you're still getting ~1980s/1990s+whatever optimizations we could add performance on a lot of the immutable stuff. It does do some of the obvious optimizations like rewriting obvious recursive algorithms to use mutability internally, but in general you're still going to pay some penalty here. 3. This one may be a bit controversial, so let me first say I deeply respect Erlang as a design, consider it to have been very far ahead of its time, and that given the general understandings of programming language theory at the time, that Erlang is a staggering accomplishment. That said, with the benefit of decades of hindsight on the design, the Erlang type system is deeply suboptimal. From what I can see, the primary purpose of the design is to ensure that you can't pass references between processes so that there's no way to modify a process' memory from another. The way it accomplished this is with a type system that has no references in it. But because this matter wasn't as well understood then as it is now, it also overcompensated with making everything immutable, removing all ability to have custom user types, and yet at the same time, having a fully dynamic type system in which there is only one type: "Erlang term". It's not necessary to kill all custom user types; you can just kill the things that make them unable to be passed across the network, which isn't everything. You don't need to make things immutable, you just need to ensure that references can't be passed. You don't need to make everything dynamically typed so that you don't have to synchronize definitions of those types; there are other ways that this can be dealt with. There's just a lot of these little things where one could finesse the result but the Erlang design hits it with a sledgehammer. There's a case to be made for that, too. A lot of the finessing would be a lot more complicated (e.g., Rust does a lot better job with managing mutability than Erlang, but look at the complication in the type system as a result; it doesn't come for free.) As language design goes, there's a lot of room for debate. But in terms of the performance impact, Erlang ends up with the worst aspects of dynamic typing on performance, and with not all that many of the benefits of dynamic typing. (You get some. There's more dynamism than initially meets the eye in Erlang. For instance, where you say lists:append(L1, L2), lists and append are just atoms. You can say L = lists, A = append, and then run L:A(L1, L2), and do conditionals on that, etc. It took me a long time to learn that. But you still miss out on a lot of the dynamism of dynamic languages, and even what there is is often harder to manage. And also, you pay in the performance for this, too.) An Erlang written for the modern era, but tuned a lot more for performance, isn't Go. But Go is probably close enough to inhibit success for any such effort; I observe that languages tend to inhibit the creation of things very close to where they are, but not the same. (Bizarrely, it seems to be easier to create something that is essentially a clone of a current language from a programming language theory perspective with an opinionated syntax gloss on it than to create a language that is mostly like another but with a couple of important PLT changes. I don't think I fully understand why this is, but the observation seems pretty solid. So, "Go + full process isolation" is unlikely to attract enough support to succeed, even though I'd personally love to see it.)
  - igouy 7y ago
    > Why doesn't it have… Perhaps because it wasn't intended to be used for everything — "What sort of problems is Erlang not particularly suitable for? … Most (all?) large systems developed using Erlang make heavy use of C for low-level code, leaving Erlang to manage the parts which tend to be complex in other languages, like controlling systems spread across several machines and implementing complex protocol logic." http://erlang.org/faq/introduction.html#idp32114576 http://erlang.org/faq/introduction.html#idp32114576
    - chrisseaton 7y ago
      This seems like an argument being made backwards. > Most (all?) large systems developed using Erlang make heavy use of C for low-level code But they use C... because Erlang is too slow. Erlang isn't slow because they use C. This is actually an argument for the need to make Erlang faster - we've got proof it's not fast enough for what they want to do! And the worst thing is that when you start to use C code it's them a self-fulfilling prophecy - you can't optimise your Erlang code because it's now lots of calls to native code. I wrote my PhD on this problem.
      - zzzcpan 7y ago
        Isn't that how all scripting languages became viable, they provided fast C extensions where it mattered the most. I.e. dropping to C for performance is by design.
        chrisseaton 7y ago
        Yes it made sense at the time. But now look at Ruby where they're trying to add a better GC and JIT because people want more speed - they're severely restricted in their options by the design of the C extension API.
        deleted 7y ago
        [deleted]
      - dragonwriter 7y ago
- lliamander 7y ago
  I appreciate the work you are doing on Loom. As someone who has done production Erlang code in the past and presently has to code in Java, I eagerly await the day that Loom becomes production ready. What are your thoughts on Pony (https://www.ponylang.io/ https://www.ponylang.io/)?
  - pron 7y ago
    I just saw a couple of talks about it. Seems very interesting (more interesting to me than Rust), but I think it's too opinionated to gain much attention in this climate.
    - lliamander 7y ago
      > Seems very interesting (more interesting to me than Rust) My sentiments as well. > I think it's too opinionated to gain much attention in this climate Possibly. Rust is pretty opinionated but it seems to be doing well. My feeling is that Pony's probably not going to "win" (i.e. be the main replacement for C/C++) but I'm hoping that, like Erlang, it's able to carve out a stable enough niche.
eternalban 7y ago
IIRC, Kilim (a Java library) performed better than Erlang in benchmarks.
verttii 7y ago
I'd say the feature of BEAM is fault-tolerant massive scalability of concurrency rather than performance/throughput. JVM as a platform lacks all the concurrency tooling of BEAM like the integrated debugging and monitoring tools. The AKKA implementation seems like a total afterthought rather than the fundamental core which it is in Erlang. Moreover, from what I understand unlike BEAM, JVM does not prioritize low consistent latency but throughput instead. I just feel they're designed for very different purposes. BEAM is most definitely not a general computing platform and would not perform well as such. In contrast, I'd imagine it'd be painful to develop concurrent stateful distributed systems on top of the JVM, regardless of the language.
- pron 7y ago
  > In contrast, I'd imagine it'd be painful to develop concurrent stateful distributed systems on top of the JVM, regardless of the language. Concurrent stateful distributed systems on the Java platform probably outnumber those on BEAM 100 to 1, but I agree that we absolutely love Erlang's relevant constructs, which is why we're bringing them to the Java platform.

by patrec 7y ago

pron 7y ago

I would be. BEAM is just a very slow platform, not just compared to Java, but also compared to JavaScript and Go. Again, that's not to say it's not very useful for some things (although many Erlang apps are really mixed Erlang/C apps because of BEAM's performance). Hey, CPython is really slow, too and yet very useful. Green threads is another matter as it's not a feature that currently exists on the Java platform (there are somewhat similar features in Java platform languages like Kotlin's coroutines), but there is currently a project (that I'm leading) to add fibers (AKA user-mode threads, AKA lightweight threads, AKA green threads) to the JVM: https://wiki.openjdk.java.net/display/loom/ https://wiki.openjdk.java.net/display/loom/

mruts 7y ago
There’s Akka. Also Scala has Future’s. Both are very similar to green threads.
chrisseaton 7y ago
It's true that BEAM has excellent concurrency, parallelism, supervision, message passing... but I don't know understand why that has to come at the expense of being terrible at everything else. Why doesn't it have even a baseline reasonable quality JIT? Why doesn't its GC use a design more recent than half a century ago? It's got big corporate backers and PhDs working on it.
- pron 7y ago
  I would imagine that's because they'd rather focus whatever resources they have on the needs and priorities of their target audience, which probably don't think performance is critical. I hope that one day TruffleBEAM+Loom would be a great platform for all those wonderful Erlang apps.
  - zambal 7y ago
    Searching for TruffleBEAM+Loom doesn't yield any meaningful results. Could you provide any info about what it is?
    - deleted 7y ago
      [deleted]
    - pron 7y ago
      It's something that I hope would exist one day. Loom is a project that I'm working on to add lightweight threads to the Java platform (https://wiki.openjdk.java.net/display/loom/ https://wiki.openjdk.java.net/display/loom/) and Truffle is a framework that allows writing fast language runtimes for the Java platform or standalone that chrisseaton is working on (https://github.com/oracle/graal/tree/master/truffle https://github.com/oracle/graal/tree/master/truffle).
- jerf 7y ago
  "but I don't know understand why that has to come at the expense of being terrible at everything else." A great deal of it is certainly just resource poverty compared to the JVM or the various JS runtimes. It may have some nice high-powered resources dedicated to it, but I'm still very comfortable with pron's assessment that the JVM has literally 100x more resources poured into it. There are also some decisions that I would consider suboptimal for performance in the original Erlang specification though: 1. It extensively uses linked lists as a data structure. It was very fashionable in functional programming at the time, but the performance impact has in relative terms gotten worse as CPUs continue to speed up relative to RAM. Erlang does recover some of this vs. other functional programming languages in that the process model tends to keep the linked list components closer together together in RAM because they'll stay in the process' arena rather than being spread arbitrarily out over RAM, so walking a linked list a couple times is at least very likely to fit well into L1, but this is still going to be a pervasive loss of performance. 2. Haskell has done a lot of work in how to reconcile immutability with performance, and I'd still say there's a penalty there. Erlang hasn't, and a lot of it wouldn't apply (Erlang is strict), so you're still getting ~1980s/1990s+whatever optimizations we could add performance on a lot of the immutable stuff. It does do some of the obvious optimizations like rewriting obvious recursive algorithms to use mutability internally, but in general you're still going to pay some penalty here. 3. This one may be a bit controversial, so let me first say I deeply respect Erlang as a design, consider it to have been very far ahead of its time, and that given the general understandings of programming language theory at the time, that Erlang is a staggering accomplishment. That said, with the benefit of decades of hindsight on the design, the Erlang type system is deeply suboptimal. From what I can see, the primary purpose of the design is to ensure that you can't pass references between processes so that there's no way to modify a process' memory from another. The way it accomplished this is with a type system that has no references in it. But because this matter wasn't as well understood then as it is now, it also overcompensated with making everything immutable, removing all ability to have custom user types, and yet at the same time, having a fully dynamic type system in which there is only one type: "Erlang term". It's not necessary to kill all custom user types; you can just kill the things that make them unable to be passed across the network, which isn't everything. You don't need to make things immutable, you just need to ensure that references can't be passed. You don't need to make everything dynamically typed so that you don't have to synchronize definitions of those types; there are other ways that this can be dealt with. There's just a lot of these little things where one could finesse the result but the Erlang design hits it with a sledgehammer. There's a case to be made for that, too. A lot of the finessing would be a lot more complicated (e.g., Rust does a lot better job with managing mutability than Erlang, but look at the complication in the type system as a result; it doesn't come for free.) As language design goes, there's a lot of room for debate. But in terms of the performance impact, Erlang ends up with the worst aspects of dynamic typing on performance, and with not all that many of the benefits of dynamic typing. (You get some. There's more dynamism than initially meets the eye in Erlang. For instance, where you say lists:append(L1, L2), lists and append are just atoms. You can say L = lists, A = append, and then run L:A(L1, L2), and do conditionals on that, etc. It took me a long time to learn that. But you still miss out on a lot of the dynamism of dynamic languages, and even what there is is often harder to manage. And also, you pay in the performance for this, too.) An Erlang written for the modern era, but tuned a lot more for performance, isn't Go. But Go is probably close enough to inhibit success for any such effort; I observe that languages tend to inhibit the creation of things very close to where they are, but not the same. (Bizarrely, it seems to be easier to create something that is essentially a clone of a current language from a programming language theory perspective with an opinionated syntax gloss on it than to create a language that is mostly like another but with a couple of important PLT changes. I don't think I fully understand why this is, but the observation seems pretty solid. So, "Go + full process isolation" is unlikely to attract enough support to succeed, even though I'd personally love to see it.)
- igouy 7y ago
  > Why doesn't it have… Perhaps because it wasn't intended to be used for everything — "What sort of problems is Erlang not particularly suitable for? … Most (all?) large systems developed using Erlang make heavy use of C for low-level code, leaving Erlang to manage the parts which tend to be complex in other languages, like controlling systems spread across several machines and implementing complex protocol logic." http://erlang.org/faq/introduction.html#idp32114576 http://erlang.org/faq/introduction.html#idp32114576
  - chrisseaton 7y ago
    This seems like an argument being made backwards. > Most (all?) large systems developed using Erlang make heavy use of C for low-level code But they use C... because Erlang is too slow. Erlang isn't slow because they use C. This is actually an argument for the need to make Erlang faster - we've got proof it's not fast enough for what they want to do! And the worst thing is that when you start to use C code it's them a self-fulfilling prophecy - you can't optimise your Erlang code because it's now lots of calls to native code. I wrote my PhD on this problem.
    - zzzcpan 7y ago
      Isn't that how all scripting languages became viable, they provided fast C extensions where it mattered the most. I.e. dropping to C for performance is by design.
      - chrisseaton 7y ago
        Yes it made sense at the time. But now look at Ruby where they're trying to add a better GC and JIT because people want more speed - they're severely restricted in their options by the design of the C extension API.
        deleted 7y ago
        [deleted]
    - dragonwriter 7y ago
lliamander 7y ago
I appreciate the work you are doing on Loom. As someone who has done production Erlang code in the past and presently has to code in Java, I eagerly await the day that Loom becomes production ready. What are your thoughts on Pony (https://www.ponylang.io/ https://www.ponylang.io/)?
- pron 7y ago
  I just saw a couple of talks about it. Seems very interesting (more interesting to me than Rust), but I think it's too opinionated to gain much attention in this climate.
  - lliamander 7y ago
    > Seems very interesting (more interesting to me than Rust) My sentiments as well. > I think it's too opinionated to gain much attention in this climate Possibly. Rust is pretty opinionated but it seems to be doing well. My feeling is that Pony's probably not going to "win" (i.e. be the main replacement for C/C++) but I'm hoping that, like Erlang, it's able to carve out a stable enough niche.

eternalban 7y ago

IIRC, Kilim (a Java library) performed better than Erlang in benchmarks.

verttii 7y ago

I'd say the feature of BEAM is fault-tolerant massive scalability of concurrency rather than performance/throughput. JVM as a platform lacks all the concurrency tooling of BEAM like the integrated debugging and monitoring tools. The AKKA implementation seems like a total afterthought rather than the fundamental core which it is in Erlang. Moreover, from what I understand unlike BEAM, JVM does not prioritize low consistent latency but throughput instead. I just feel they're designed for very different purposes. BEAM is most definitely not a general computing platform and would not perform well as such. In contrast, I'd imagine it'd be painful to develop concurrent stateful distributed systems on top of the JVM, regardless of the language.

pron 7y ago
> In contrast, I'd imagine it'd be painful to develop concurrent stateful distributed systems on top of the JVM, regardless of the language. Concurrent stateful distributed systems on the Java platform probably outnumber those on BEAM 100 to 1, but I agree that we absolutely love Erlang's relevant constructs, which is why we're bringing them to the Java platform.