6 ms·

My take on it is that I like most of the design choices they've made (Been tinkering on a similar dynamic runtime before). With a dynamic language like Erlang

My take on it is that I like most of the design choices they've made (Been tinkering on a similar dynamic runtime before). With a dynamic language like Erlang (as with JS,Lua,etc) the main benefits won't come from better register allocation,DCE,etc that is the primary reason one would pick LLVM but rather just getting rid of the interpreter overhead (and later the oppurtunity to do some higher level type dependent optimizations that will have a higher impact than register allocation and instruction selection). Type dependant stuff why LLVM is ill suited IMHO and why you only see LLVM being mentioned as the highest optimizataion level by for example JavaScriptCore(Safari) when the runtime hasn't deoptimized code and is pretty certain of the types that will be used. Also they mentioned the complexity of maintaining interpreter/JITed jumps and I'm not surprised since i remember reading some paper about one of their earlier attempts and they were maintaining dual stacks with quite complex cross-boundary jumps. Some might mention that V8 moved away from the always-JIT paradigm to save on memory and startup time but since Erlang is mostly used in servers i think they might see this as a good compromise.

chrisseaton 6y ago
> that will have a higher impact than register allocation and instruction selection The most powerful JIT compiler that I have worked with - Graal - generates code that often on the face of it seems puzzlingly inefficient in terms of registers allocated and instructions selected. Turns out maybe it's another thing that's not as important as all the text books say? The important bit is removing object allocating, removing boxing, deep inlining, and specialisation... when you've done all that work the exact registers and instructions don't seem to really make a huge difference.
- Rochus 6y ago
  Are there any papers or articles about the mentioned findings?
  - chrisseaton 6y ago
    About what findings, sorry? Linear Scan is an example I reach for when I talk about what parts of the compiler are really important, if that's what you mean. http://web.cs.ucla.edu/~palsberg/course/cs132/linearscan.pdf http://web.cs.ucla.edu/~palsberg/course/cs132/linearscan.pdf > The linear scan algorithm is considerably faster than algorithms based on graph coloring, is simple to implement, and results in code that is almost as efficient as that obtained using more complex and time-consuming register allocators based on graph coloring. Also things like escape analysis and inlining are often called 'the mothers of optimisation' because they fundamentally enable so many other optimisations. Not sure there's really a citation for it but I doubt anyone would dispute. I wrote about the impact on optimising Ruby. https://chrisseaton.com/truffleruby/pushing-pixels/ https://chrisseaton.com/truffleruby/pushing-pixels/
    - Rochus 6y ago
      > About what findings, sorry? These findings: The important bit is removing object allocating, removing boxing, deep inlining, and specialisation... when you've done all that work the exact registers and instructions don't seem to really make a huge difference. > Linear Scan is an example I reach for when I talk about what parts of the compiler are really important Sure, but you just said that register allocation is not a relevant factor compared to other achievements when using Gral VM. Or did I get that wrong?
      - chrisseaton 6y ago
        > These findings: Right - didn't I cover those? I gave the example that Graal is extremely powerful with a huge amount of money and effort behind it, but doesn't really bother at all with serious register allocation or clever instruction selection as suggested you should in text books. It uses the most basic algorithm and doesn't see any need to do any more, even when they're still keen on tiny improvements to the output. It just doesn't seem to be worth it. But it does put a lot of effort into object allocation, boxing, inlining, specialisation, etc. So those seem in practice to be worth it.
        6y ago
- MaxBarraclough 6y ago
  Could that be due to the cleverness of modern heavyweight CPUs, with techniques like register renaming? Would things change if you used less sophisticated processors?
  - chrisseaton 6y ago
    This is the explanation I get when I dig into these things, yes. For example I'll see what seems to my less-experienced eyes some completely redundant 'mov' instructions shifting things around. I'll ask 'should we fix this' and the response is usually 'doesn't really matter it's basically free'.
    - MaxBarraclough 6y ago
      Interesting, so (to a rough approximation) the CPU is smart enough to boil off many 'shallow' inefficiencies. Doesn't cache behaviour still matter though? I'd have thought code-compactness would still be worth carefully optimising for.
      - chrisseaton 6y ago
        > Doesn't cache behaviour still matter though? I'd have thought code-compactness would still be worth carefully optimising for. Well that's partly why I'm still surprised, but I think for a lot of dynamic languages there's such a huge volume of code anyway... that these little 'mov's are pretty insignificant. Removing them would have a down-side - extra compilation time.
      - beagle3 6y ago
        yes and no. Most instructions have 3-20 cycle latency. A lot of what CPU do on many programs is wait for an instruction to complete so they can use its result to chase some pointer - and in the mean time, find an instruction that doesn't depend on it to execute. So many "simple" moves and register renames are practically free, as they happen in these waiting-for-result slots. HPC code is most definitely not like that - but it's only found in specialized parts of software (game rendering paths, maths stuff, some NumPy code, etc). But most code spends most of its time with the CPU waiting.
- KMag 6y ago
  Even with inlining and escape analysis to remove unnecessary boxing, unboxing, and heap allocations, most Java programs still do a lot of pointer chasing. All of that (often poor locality) memory traffic tends to hide the overhead of unnecessary register moves and even some unnecessary register spills.
- higerordermap 6y ago
  How fast is code generated by graal? In languages like Java w/o value types & virtual methods everywhere, interprocedural analysis is very important. Even otherwise I believe interprocedural optimizations are more important than small register fiddling or cute integer arithmetic tricks. But when you reach a dead-end in optimized code, all these things matter. Sure, it may not matter for Erlang / Elixir on server.
nickjj 6y ago
> Erlang is mostly used in servers i think they might see this as a good compromise. I'm not sure what the differences will be but start up time is still a very big deal for web servers. For example I can start up a non-Dockerized Python / gunicorn app with 8,000 lines of code and 50+ package dependencies in about 400ms on a $5 a month DigitalOcean server without doing any type of optimizations. For someone who just wants to deploy a project to 1 server that means there's only 400ms of down time when I restart my server. If a runtime takes let's say 10 seconds to start up, that could be enough of a deterrent to not use it if you know you're going down the route of a single server deploy.
- cmrdporcupine 6y ago
  I don't think we're talking "start time" so much as "warm up time"; if I understand these things correctly it's likely the VM would start almost immediately, but would take a little bit of time for hot code paths to JIT and become highly performant. I don't think that would be much of a concern in your example.
  - lawik 6y ago
    From my conversation with them on Elixir Mix I don't believe warm-up is the issue. In this case the JIT is simpler than that. But the BEAM VM isn't the quickest starter in the world.
- dnautics 6y ago
  > For someone who just wants to deploy a project to 1 server that means there's only 400ms of down time when I restart my server. Erlang already gives you robust primitives to do things like blue-green deploys, and even graceful transfer of server state even across versioning changes. If downtime between releases is an something you care about, you should use those, and it's likely that your downtime can be in the microseconds range regardless of the vm startup latency.
  - nickjj 6y ago
    I don't think hot reloads in Elixir are used that often for web apps due to how much complexity / state changes need to take place. I'm basing that on replies I've seen from a lot of folks in the forums who say it's not really worth the trouble due to how much can go wrong and how involved it would be to do such a thing. Which blue-green deploy primitives are you referencing btw?
kannanvijayan 6y ago
> Some might mention that V8 moved away from the always-JIT They did, but their always-JIT was a bit heavyweight. They generated full method-jit code on first invocation at first. This was a massive improvement over the existing state of the art, but it was also relatively early in the whole development of the "fast JS" ecosystem we live in now. The _size_ of JS grew, and the size of functions grew, and the complexity of logic grew. The amount of cold and run-once code grew. That issue was brought into focus by the growth of the webapp space and the size of the payloads. In this case (erlang folks) - they're going about the whole thing in a very good way. Their "jit-everything" is actually "jit one instruction at a time", which is _amazingly_ perceptive of the challenges people in other JIT teams have faced (e.g. JS JIT teams). What they're doing is tightly scoped, easy to bootstrap and test with fallbacks to VM-calls into the VM for slowpaths or complicated stuff. That's a solid base you can slowly layer with higher tiers later if it matters. They're keeping their abstraction layers strong by having a well-specified bytecode system, and hopefully they will strive to keep it relatively independent of the runtime, and avoid leakages of runtime semantics into instructions. I was personally very impressed by their description of their approach and motivation behind each decision.

chrisseaton 6y ago

> that will have a higher impact than register allocation and instruction selection The most powerful JIT compiler that I have worked with - Graal - generates code that often on the face of it seems puzzlingly inefficient in terms of registers allocated and instructions selected. Turns out maybe it's another thing that's not as important as all the text books say? The important bit is removing object allocating, removing boxing, deep inlining, and specialisation... when you've done all that work the exact registers and instructions don't seem to really make a huge difference.

Rochus 6y ago
Are there any papers or articles about the mentioned findings?
- chrisseaton 6y ago
  About what findings, sorry? Linear Scan is an example I reach for when I talk about what parts of the compiler are really important, if that's what you mean. http://web.cs.ucla.edu/~palsberg/course/cs132/linearscan.pdf http://web.cs.ucla.edu/~palsberg/course/cs132/linearscan.pdf > The linear scan algorithm is considerably faster than algorithms based on graph coloring, is simple to implement, and results in code that is almost as efficient as that obtained using more complex and time-consuming register allocators based on graph coloring. Also things like escape analysis and inlining are often called 'the mothers of optimisation' because they fundamentally enable so many other optimisations. Not sure there's really a citation for it but I doubt anyone would dispute. I wrote about the impact on optimising Ruby. https://chrisseaton.com/truffleruby/pushing-pixels/ https://chrisseaton.com/truffleruby/pushing-pixels/
  - Rochus 6y ago
    > About what findings, sorry? These findings: The important bit is removing object allocating, removing boxing, deep inlining, and specialisation... when you've done all that work the exact registers and instructions don't seem to really make a huge difference. > Linear Scan is an example I reach for when I talk about what parts of the compiler are really important Sure, but you just said that register allocation is not a relevant factor compared to other achievements when using Gral VM. Or did I get that wrong?
    - chrisseaton 6y ago
      > These findings: Right - didn't I cover those? I gave the example that Graal is extremely powerful with a huge amount of money and effort behind it, but doesn't really bother at all with serious register allocation or clever instruction selection as suggested you should in text books. It uses the most basic algorithm and doesn't see any need to do any more, even when they're still keen on tiny improvements to the output. It just doesn't seem to be worth it. But it does put a lot of effort into object allocation, boxing, inlining, specialisation, etc. So those seem in practice to be worth it.
      - 6y ago
MaxBarraclough 6y ago
Could that be due to the cleverness of modern heavyweight CPUs, with techniques like register renaming? Would things change if you used less sophisticated processors?
- chrisseaton 6y ago
  This is the explanation I get when I dig into these things, yes. For example I'll see what seems to my less-experienced eyes some completely redundant 'mov' instructions shifting things around. I'll ask 'should we fix this' and the response is usually 'doesn't really matter it's basically free'.
  - MaxBarraclough 6y ago
    Interesting, so (to a rough approximation) the CPU is smart enough to boil off many 'shallow' inefficiencies. Doesn't cache behaviour still matter though? I'd have thought code-compactness would still be worth carefully optimising for.
    - chrisseaton 6y ago
      > Doesn't cache behaviour still matter though? I'd have thought code-compactness would still be worth carefully optimising for. Well that's partly why I'm still surprised, but I think for a lot of dynamic languages there's such a huge volume of code anyway... that these little 'mov's are pretty insignificant. Removing them would have a down-side - extra compilation time.
    - beagle3 6y ago
      yes and no. Most instructions have 3-20 cycle latency. A lot of what CPU do on many programs is wait for an instruction to complete so they can use its result to chase some pointer - and in the mean time, find an instruction that doesn't depend on it to execute. So many "simple" moves and register renames are practically free, as they happen in these waiting-for-result slots. HPC code is most definitely not like that - but it's only found in specialized parts of software (game rendering paths, maths stuff, some NumPy code, etc). But most code spends most of its time with the CPU waiting.
KMag 6y ago
Even with inlining and escape analysis to remove unnecessary boxing, unboxing, and heap allocations, most Java programs still do a lot of pointer chasing. All of that (often poor locality) memory traffic tends to hide the overhead of unnecessary register moves and even some unnecessary register spills.
higerordermap 6y ago
How fast is code generated by graal? In languages like Java w/o value types & virtual methods everywhere, interprocedural analysis is very important. Even otherwise I believe interprocedural optimizations are more important than small register fiddling or cute integer arithmetic tricks. But when you reach a dead-end in optimized code, all these things matter. Sure, it may not matter for Erlang / Elixir on server.

nickjj 6y ago

> Erlang is mostly used in servers i think they might see this as a good compromise. I'm not sure what the differences will be but start up time is still a very big deal for web servers. For example I can start up a non-Dockerized Python / gunicorn app with 8,000 lines of code and 50+ package dependencies in about 400ms on a $5 a month DigitalOcean server without doing any type of optimizations. For someone who just wants to deploy a project to 1 server that means there's only 400ms of down time when I restart my server. If a runtime takes let's say 10 seconds to start up, that could be enough of a deterrent to not use it if you know you're going down the route of a single server deploy.

cmrdporcupine 6y ago
I don't think we're talking "start time" so much as "warm up time"; if I understand these things correctly it's likely the VM would start almost immediately, but would take a little bit of time for hot code paths to JIT and become highly performant. I don't think that would be much of a concern in your example.
- lawik 6y ago
  From my conversation with them on Elixir Mix I don't believe warm-up is the issue. In this case the JIT is simpler than that. But the BEAM VM isn't the quickest starter in the world.
dnautics 6y ago
> For someone who just wants to deploy a project to 1 server that means there's only 400ms of down time when I restart my server. Erlang already gives you robust primitives to do things like blue-green deploys, and even graceful transfer of server state even across versioning changes. If downtime between releases is an something you care about, you should use those, and it's likely that your downtime can be in the microseconds range regardless of the vm startup latency.
- nickjj 6y ago
  I don't think hot reloads in Elixir are used that often for web apps due to how much complexity / state changes need to take place. I'm basing that on replies I've seen from a lot of folks in the forums who say it's not really worth the trouble due to how much can go wrong and how involved it would be to do such a thing. Which blue-green deploy primitives are you referencing btw?

kannanvijayan 6y ago

> Some might mention that V8 moved away from the always-JIT They did, but their always-JIT was a bit heavyweight. They generated full method-jit code on first invocation at first. This was a massive improvement over the existing state of the art, but it was also relatively early in the whole development of the "fast JS" ecosystem we live in now. The _size_ of JS grew, and the size of functions grew, and the complexity of logic grew. The amount of cold and run-once code grew. That issue was brought into focus by the growth of the webapp space and the size of the payloads. In this case (erlang folks) - they're going about the whole thing in a very good way. Their "jit-everything" is actually "jit one instruction at a time", which is _amazingly_ perceptive of the challenges people in other JIT teams have faced (e.g. JS JIT teams). What they're doing is tightly scoped, easy to bootstrap and test with fallbacks to VM-calls into the VM for slowpaths or complicated stuff. That's a solid base you can slowly layer with higher tiers later if it matters. They're keeping their abstraction layers strong by having a well-specified bytecode system, and hopefully they will strive to keep it relatively independent of the runtime, and avoid leakages of runtime semantics into instructions. I was personally very impressed by their description of their approach and motivation behind each decision.