4 ms·

Erlang's semantics are deeply intertwined with the unique things the Erlang runtime does. For example, the Erlang abstract machine does straight-line non-preem

by derefr 5y ago

Erlang's semantics are deeply intertwined with the unique things the Erlang runtime does. For example, the Erlang abstract machine does straight-line non-preemptable atomic execution within bytecode basic-blocks, with reduction-checking for yield exactly/only at stack-frame manipulation points (i.e. call/ret/tail-call.) Those points are guaranteed to occur after O(1) reductions, because of an ISA design that contains no unbounded local looping primitives — i.e. no way to encode relative jumps with negative offsets. (Note that this design requirement — and not any functional-programming ideal — is why Erlang uses tail-calls for looping. It has to; there's no other way to do loops given the ISA constraints!) This atomicity of bytecode basic-blocks is what guarantees that actors can be hard-killed without corrupting the abstract-machine scheduler they run on (they die at their next yield-point, with the scheduler in a coherent state). It's a fundamental difference between Erlang scheduling and JVM scheduling. The JVM doesn't have this atomicity, and so you can't hard-kill a Java thread without corrupting the JVM. Instead, you can only softly "interrupt" threads — sending them a special "please die" signal they have to explicitly check for. This means that JVM languages can't support anything like Erlang's process links — i.e. JVM concurrency frameworks can't propagate failure downwards through a supervision hierarchy in a way that actually releases resources from long-running CPU-bound sub-tasks. This in turn means you can't reliably bound resource usage under high-concurrency scenarios; which means that, essentially, all the things that people get excited about adding to Java with Akka, Loom, etc. don't actually do much to help the use-cases they attempt to address. This last is personal experience, by the way. My company develops backend server software in both Erlang (Elixir) and Java. We actually tried Loom as a way of fixing some of the robustness-under-concurrency problems with the JVM; but the problems are much more fundamental than just adding features like virtual threads can resolve.

kaba0 5y ago
I didn’t meant it as “only” a library, but as a fork. I still don’t necessarily see why it would be an insurmountable problem to fork the OpenJDK project and add atomic execution to it (do note that I’m sure it has the necessary mechanism for that — JIT deoptimizations pretty much require dropping the previously calculated results, so with some change it could be doable). It wouldn’t be a soft fork and would still require plenty of resources no doubt, but I still feel like going this direction will better utilize the insane dev-hours that went into the OpenJDK’s stellar performance, than trying to fix the performance issues of Erlang. But do take everything I said with a huge grain of salt, as I am neither an OpenJDK contributor and as you can see don’t know much about the Erlang ecosystem. And thank you for the very informative look behind the scenes.
- di4na 5y ago
  Yeah no. The problem here is that a huge part of said jit performance on jvm depends on the assumption that you can elide information that the schedulers and tracing would need. Reverting that would need to revert nearly all advanced optimisations plus to totally change the memory model and the GC. At this point, you have already lost all benefits.
dszoboszlay 5y ago
This is not exactly how the Beam VM works. > an ISA design that contains no unbounded local looping primitives Actually, most of the Beam instructions are non-O(1). For example since integers are unbounded in Erlang even simple arithmetic (+, -, etc.) may turn out to be a non-constant time operation (even though most of the time your integers will fit into a single machine word, so it'd rarely be a problem). But there's also a built-in for appending lists (++), which obviously contains an unbounded loop in it. The solution is that these instructions are written so that they do work on chunks of data (e.g. ++ may process 1000 elements of a list in a chunk) after which they increment the reduction counter and possibly schedule out the process. > there's no other way to do loops given the ISA constraints The Beam ISA allows looping. You can even write a hand-crafted loop that will never increment the reduction counter and thus will deadlock a scheduler. But the Erlang compiler will never generate such a loop for you. On the ISA level there are only labels where you can jump to. You can jump forwards and backwards. So you could implement a language that offers loops and still compiles to Beam. However, since jumps don't increment the reduction counter, you would either risk your loops breaking the fair scheduling of processes, or you would have to ensure that the loop body contains an operation that increments the reduction counter and allows the scheduler to suspend the process. > This atomicity of bytecode basic-blocks is what guarantees that actors can be hard-killed without corrupting the abstract-machine scheduler they run on Well, it is of course important that you don't interrupt the scheduler at an arbitrary point, midway executing an opcode. But there are no atomically executed bytecode blocks. Actors are free to kill not because they would run their code in uninterrupted atomic blocks, but because they don't share state (their heap) with each other. So if you have an actor that holds e.g. a binary tree, and it is half way into inserting a value into the binary tree when you kill it, it may leave the binary tree in an inconsistent state, but that doesn't matter, because no one else have access to this data structure: it lives on this process' own heap. When processes use shared resources (such as ETS tables, files or a gen_server process) and they are killed, they may very well leave that shared resource in an inconsistent state, just not on the VM layer, but on the application logic layer. So the file will still be usable as a file, but it may contain corrupted data for example. > The JVM doesn't have this atomicity, and so you can't hard-kill a Java thread without corrupting the JVM. Instead, you can only softly "interrupt" threads. If you would port Erlang to the JVM, that would be the least of your problems. The compiler could just insert code to check for these signals every now and then. I go further: if you'd run Erlang code (and only Erlang code) on the JVM, it wouldn't even matter that you don't have separate heaps. Every process would only use a separate part of the shared heap, so they couldn't tip on each other's toe. The GC could take care of the rest as usual. I think there are two real issues with porting to the JVM: * Mapping an Erlang process to an OS thread would only work up to some reasonably low number of Erlang processes. After that you'd have to switch to a green thread model with schedulers, which is a lot of work to implement. * The Beam put a lot of effort into making the VM scale well to a lot of schedulers. Things like how to implement a message box where 100+ schedulers can concurrently push messages to. You'd probably have to implement similar optimisations for the data structures you'd use for message boxes, ETS tables etc. on the JVM too.

by derefr 5y ago

kaba0 5y ago

I didn’t meant it as “only” a library, but as a fork. I still don’t necessarily see why it would be an insurmountable problem to fork the OpenJDK project and add atomic execution to it (do note that I’m sure it has the necessary mechanism for that — JIT deoptimizations pretty much require dropping the previously calculated results, so with some change it could be doable). It wouldn’t be a soft fork and would still require plenty of resources no doubt, but I still feel like going this direction will better utilize the insane dev-hours that went into the OpenJDK’s stellar performance, than trying to fix the performance issues of Erlang. But do take everything I said with a huge grain of salt, as I am neither an OpenJDK contributor and as you can see don’t know much about the Erlang ecosystem. And thank you for the very informative look behind the scenes.

di4na 5y ago
Yeah no. The problem here is that a huge part of said jit performance on jvm depends on the assumption that you can elide information that the schedulers and tracing would need. Reverting that would need to revert nearly all advanced optimisations plus to totally change the memory model and the GC. At this point, you have already lost all benefits.

dszoboszlay 5y ago

This is not exactly how the Beam VM works. > an ISA design that contains no unbounded local looping primitives Actually, most of the Beam instructions are non-O(1). For example since integers are unbounded in Erlang even simple arithmetic (+, -, etc.) may turn out to be a non-constant time operation (even though most of the time your integers will fit into a single machine word, so it'd rarely be a problem). But there's also a built-in for appending lists (++), which obviously contains an unbounded loop in it. The solution is that these instructions are written so that they do work on chunks of data (e.g. ++ may process 1000 elements of a list in a chunk) after which they increment the reduction counter and possibly schedule out the process. > there's no other way to do loops given the ISA constraints The Beam ISA allows looping. You can even write a hand-crafted loop that will never increment the reduction counter and thus will deadlock a scheduler. But the Erlang compiler will never generate such a loop for you. On the ISA level there are only labels where you can jump to. You can jump forwards and backwards. So you could implement a language that offers loops and still compiles to Beam. However, since jumps don't increment the reduction counter, you would either risk your loops breaking the fair scheduling of processes, or you would have to ensure that the loop body contains an operation that increments the reduction counter and allows the scheduler to suspend the process. > This atomicity of bytecode basic-blocks is what guarantees that actors can be hard-killed without corrupting the abstract-machine scheduler they run on Well, it is of course important that you don't interrupt the scheduler at an arbitrary point, midway executing an opcode. But there are no atomically executed bytecode blocks. Actors are free to kill not because they would run their code in uninterrupted atomic blocks, but because they don't share state (their heap) with each other. So if you have an actor that holds e.g. a binary tree, and it is half way into inserting a value into the binary tree when you kill it, it may leave the binary tree in an inconsistent state, but that doesn't matter, because no one else have access to this data structure: it lives on this process' own heap. When processes use shared resources (such as ETS tables, files or a gen_server process) and they are killed, they may very well leave that shared resource in an inconsistent state, just not on the VM layer, but on the application logic layer. So the file will still be usable as a file, but it may contain corrupted data for example. > The JVM doesn't have this atomicity, and so you can't hard-kill a Java thread without corrupting the JVM. Instead, you can only softly "interrupt" threads. If you would port Erlang to the JVM, that would be the least of your problems. The compiler could just insert code to check for these signals every now and then. I go further: if you'd run Erlang code (and only Erlang code) on the JVM, it wouldn't even matter that you don't have separate heaps. Every process would only use a separate part of the shared heap, so they couldn't tip on each other's toe. The GC could take care of the rest as usual. I think there are two real issues with porting to the JVM: * Mapping an Erlang process to an OS thread would only work up to some reasonably low number of Erlang processes. After that you'd have to switch to a green thread model with schedulers, which is a lot of work to implement. * The Beam put a lot of effort into making the VM scale well to a lot of schedulers. Things like how to implement a message box where 100+ schedulers can concurrently push messages to. You'd probably have to implement similar optimisations for the data structures you'd use for message boxes, ETS tables etc. on the JVM too.