4 ms·

Deegen: A JIT-Capable VM Generator for Dynamic Languages

46 points by igouy 2y ago

igouy 2y ago
"the disassembly of the Deegen-generated interpreter, baseline JIT, and the generated JIT code rivals the assembly code hand-written by assembly experts in state-of-the-art VMs."
- Rochus 2y ago
  Apparently they compare their JIT with the LuaJIT interpreter. I would be impressed if their JIT was 30% faster on average than LuaJIT in JIT mode. The Graal/Truffle generated VMs are much faster (see e.g. http://software.rochus-keller.ch/awfy-bun-summary.ods http://software.rochus-keller.ch/awfy-bun-summary.ods).
  - tekknolagi 2y ago
    No, they also compare JIT to JIT and theirs is 30% slower. But it's only a baseline JIT and they'll have optimizing soon (tm).
    - mananaysiempre 2y ago
      It’s also a completely different class of JIT: a method-at-a-time one, not a tracing one. As I’ve mentioned in a thread some time ago, this is a very impressive project that is a JIT for Lua, but it has so little to do with LuaJIT’s architecture otherwise that calling it LuaJIT Remake feels actively misleading. It’s SpiderMonkey for Lua, if anything.
    - Rochus 2y ago
      Ok, apparently I misinterpreted the text; it would be easier to unserstand if they used factors instead of percentages. Actually it's pretty close to LuaJIT in JIT performance, which calls into question both the use of manual assembler programming and the huge effort and complexity of the handmade tracing JIT. However, it is not yet clear to me whether there are other factors, so that, for example, the results on 32-bit architectures would be better for LuaJIT. Or maybe they profit of the fact that there are still a lot of operations in LuaJIT not supported by the JIT (such as FNEW, which is detrimental for all applications depending on closures), and their baseline JIT supports them in contrast.
  - mananaysiempre 2y ago
    > Graal/Truffle generated VMs are much faster (see e.g. [link]). Faster than what? I don’t see any mention of any kind of Lua in that table or in the page it mentions. It’d be awesome[1] if Graal could outdo LuaJIT on Lua, and I was initially excited to learn that it did, but I don’t see anything about that there. [1] Or as awesome as it’s possible to be for something that Oracle evidently intends to patent to the gills, anyway.
    - Rochus 2y ago
      You can e.g. use Node.js as a reference and compare with these results: https://github.com/rochus-keller/Oberon/blob/master/testcases/Are-we-fast-yet/Are-we-fast-yet_results_linux.pdf https://github.com/rochus-keller/Oberon/blob/master/testcase.... LuaJIT with JIT on is usually factor two slower than Node.js and factor five slower than C/C++ in my measurements.
Rochus 2y ago
> We implement LuaJIT Remake (LJR)[...] using Deegen. Across 44 benchmarks, LJR's interpreter is on average 179% faster than the official PUC Lua interpreter, and 31% faster than LuaJIT's interpreter. Well, LuaJIT in JIT mode is about factor 3 faster on average than LuaJIT in interpreter mode (depending on the benchmark up to ten times). And LuaJIT in JIT mode is e.g. factor 8 faster on average than PUC Lua 5.1 (see e.g. http://software.rochus-keller.ch/are-we-fast-yet_Lua_results_2024-06-20.pdf http://software.rochus-keller.ch/are-we-fast-yet_Lua_results... for more information). So if Degen is factor 2 faster than PUC Lua or factor 1.3 faster than the LuaJIT interpreter, this is not very impressive. But since the LuaJIT interpreter is written in assembler, we might conclude that the speed-up of a manual assembler implementation compared to a generated interpreter is about 30%. Therefore it's no longer worth the effort to implement an interpreter in assembler (even less if we consider cross-platform migration costs). But on the other hand the Degen generated VM is significantly slower than e.g. the Mono VM or CoreCLR in JIT mode (see e.g. https://github.com/rochus-keller/Oberon/blob/master/testcases/Are-we-fast-yet/Are-we-fast-yet_results_linux.pdf https://github.com/rochus-keller/Oberon/blob/master/testcase...).
- mananaysiempre 2y ago
  > we might conclude that the speed-up of a manual assembler implementation compared to a generated interpreter is about 30%[, t]herefore it's no longer worth the effort to implement an interpreter in assembler You got that backwards. The paper reports Deegen’s generated interpreter is faster than LuaJIT’s handwritten one by 30%. That’s actually pretty impressive—and pretty impressively straightforwardly achieved[1], TL;DR: instruction dispatch via tail calls avoids the pessimized register allocation that you get for a huge monolithic interpreter loop. [1] https://sillycross.github.io/2022/11/22/2022-11-22/ https://sillycross.github.io/2022/11/22/2022-11-22/
  - fweimer 2y ago
    They still have register allocation issues: > Register shuffling to fulfill C calling convention when making a runtime call. Not sure how common is that in their benchmarks because it's tempting to handle everything frequently used as bytecode.

46 points by igouy 2y ago

"the disassembly of the Deegen-generated interpreter, baseline JIT, and the generated JIT code rivals the assembly code hand-written by assembly experts in state-of-the-art VMs."

Rochus 2y ago
Apparently they compare their JIT with the LuaJIT interpreter. I would be impressed if their JIT was 30% faster on average than LuaJIT in JIT mode. The Graal/Truffle generated VMs are much faster (see e.g. http://software.rochus-keller.ch/awfy-bun-summary.ods http://software.rochus-keller.ch/awfy-bun-summary.ods).
- tekknolagi 2y ago
  No, they also compare JIT to JIT and theirs is 30% slower. But it's only a baseline JIT and they'll have optimizing soon (tm).
  - mananaysiempre 2y ago
    It’s also a completely different class of JIT: a method-at-a-time one, not a tracing one. As I’ve mentioned in a thread some time ago, this is a very impressive project that is a JIT for Lua, but it has so little to do with LuaJIT’s architecture otherwise that calling it LuaJIT Remake feels actively misleading. It’s SpiderMonkey for Lua, if anything.
  - Rochus 2y ago
    Ok, apparently I misinterpreted the text; it would be easier to unserstand if they used factors instead of percentages. Actually it's pretty close to LuaJIT in JIT performance, which calls into question both the use of manual assembler programming and the huge effort and complexity of the handmade tracing JIT. However, it is not yet clear to me whether there are other factors, so that, for example, the results on 32-bit architectures would be better for LuaJIT. Or maybe they profit of the fact that there are still a lot of operations in LuaJIT not supported by the JIT (such as FNEW, which is detrimental for all applications depending on closures), and their baseline JIT supports them in contrast.
- mananaysiempre 2y ago
  > Graal/Truffle generated VMs are much faster (see e.g. [link]). Faster than what? I don’t see any mention of any kind of Lua in that table or in the page it mentions. It’d be awesome[1] if Graal could outdo LuaJIT on Lua, and I was initially excited to learn that it did, but I don’t see anything about that there. [1] Or as awesome as it’s possible to be for something that Oracle evidently intends to patent to the gills, anyway.
  - Rochus 2y ago
    You can e.g. use Node.js as a reference and compare with these results: https://github.com/rochus-keller/Oberon/blob/master/testcases/Are-we-fast-yet/Are-we-fast-yet_results_linux.pdf https://github.com/rochus-keller/Oberon/blob/master/testcase.... LuaJIT with JIT on is usually factor two slower than Node.js and factor five slower than C/C++ in my measurements.

Rochus 2y ago

> We implement LuaJIT Remake (LJR)[...] using Deegen. Across 44 benchmarks, LJR's interpreter is on average 179% faster than the official PUC Lua interpreter, and 31% faster than LuaJIT's interpreter. Well, LuaJIT in JIT mode is about factor 3 faster on average than LuaJIT in interpreter mode (depending on the benchmark up to ten times). And LuaJIT in JIT mode is e.g. factor 8 faster on average than PUC Lua 5.1 (see e.g. http://software.rochus-keller.ch/are-we-fast-yet_Lua_results_2024-06-20.pdf http://software.rochus-keller.ch/are-we-fast-yet_Lua_results... for more information). So if Degen is factor 2 faster than PUC Lua or factor 1.3 faster than the LuaJIT interpreter, this is not very impressive. But since the LuaJIT interpreter is written in assembler, we might conclude that the speed-up of a manual assembler implementation compared to a generated interpreter is about 30%. Therefore it's no longer worth the effort to implement an interpreter in assembler (even less if we consider cross-platform migration costs). But on the other hand the Degen generated VM is significantly slower than e.g. the Mono VM or CoreCLR in JIT mode (see e.g. https://github.com/rochus-keller/Oberon/blob/master/testcases/Are-we-fast-yet/Are-we-fast-yet_results_linux.pdf https://github.com/rochus-keller/Oberon/blob/master/testcase...).

mananaysiempre 2y ago
> we might conclude that the speed-up of a manual assembler implementation compared to a generated interpreter is about 30%[, t]herefore it's no longer worth the effort to implement an interpreter in assembler You got that backwards. The paper reports Deegen’s generated interpreter is faster than LuaJIT’s handwritten one by 30%. That’s actually pretty impressive—and pretty impressively straightforwardly achieved[1], TL;DR: instruction dispatch via tail calls avoids the pessimized register allocation that you get for a huge monolithic interpreter loop. [1] https://sillycross.github.io/2022/11/22/2022-11-22/ https://sillycross.github.io/2022/11/22/2022-11-22/
- fweimer 2y ago
  They still have register allocation issues: > Register shuffling to fulfill C calling convention when making a runtime call. Not sure how common is that in their benchmarks because it's tempting to handle everything frequently used as bytecode.