3 ms·

The problem with the approach is that it's not real JIT-compilation, it's just assembly templates with basic substitutions. By not using LLVM, you're missing a

The problem with the approach is that it's not real JIT-compilation, it's just assembly templates with basic substitutions. By not using LLVM, you're missing all the optimizations it does.

mort96 2mo ago
A non-optimizing compiler is a real compiler.
- mgaunard 2mo ago
  People do not call assemblers compilers, and yet, it's doing most of what the "JIT compiler" is doing.
IshKebab 2mo ago
This absolutely is real JIT compilation. Copy and patch is a very well known JIT compilation technique.
the-lazy-guy 2mo ago
This is absolutely a JIT-compiler. It compiles code into machine code. This is a surprisingly efficient way to get noticeable speedup relative to interpretation. Also it is much safer than proper optimising compiler. Say ebpf jit-compiler functions very similarly, because it is fast and _secure_ way to jit. (well, there's a bit of cheating because before emitting bpf bytecode it goes through gcc/clang pipeline). LLVM is a large dependency if you need to JIT. There are plenty of smaller (and much faster) alternatives which are much better fit for smaller projects. Larger projects usually roll out their own jit-pipeline because they can integrate better with the source language/interpreter and apply tricks LLVM is not well suited to (say, LLVM is not great at deoptimisation). I think only Julia is really a heavy user of LLVM JIT, also it is known for extremely slow repl from time to time.
- mort96 2mo ago
  To back up the "surprisingly efficient way to get a speed-up" thing: I once wrote a toy compiler, without an optimizer and without even a register allocator (so all variables lived in stack memory). In my test benchmarks it was roughly 4x slower than Clang at -O3, IIRC. That's not exactly blazing fast for a low level C-like language, but it's not bad. It's infinitely faster than what I've ever gotten a toy interpreter to be.
bastawhiz 2mo ago
By choosing LLVM you're also taking a serious latency hit, and potentially burning a ton of CPU cycles on optimizations that will never apply. JSC, for instance, implemented LLVM for FTLJIT (which is where many of the JS bits jangling around in LLVM originated from), but it was only useful for code that was highly likely to benefit from the optimizations because of the high cost of compilation. The webkit folks have since ripped out LLVM and replaced it with their own specialized JIT, which is essentially just what's demonstrated here (with some optimization passes).
RetroTechie 2mo ago
> By not using LLVM, you're missing all the optimizations it does And yet, a good portion of software that runs today's world is written in scripting languages & executed using interpreters. Which is okay! Imho: multiply [# of users] with [how often each user sees that software's effect] and [how much that contributes to the overall user experience], then you get a ballpark idea of how much $$/effort is worth spending on optimization. In other words: for a one-off, don't bother. But as usercount, frequency of use by individual users, poor UX or RAM/CPU consumption goes up, progress from script -> compiled -> optimizing compiler -> (if necessary) hand-optimized assembly as needed. And of course consider high-level design, data structures, algorithms etc in that process. A change there might be more effective than a switch from interpreted -> optimizing compiler. "Developer time" should not factor into that much (again: imho) unless users=developers. Thoughtlessly putting every change through a (slow?) pipeline that does 'random' toolbox-of-optimizations without need, is wasteful. Apply that toolbox as needed while keeping the above in mind.
jagged-chisel 2mo ago
Sounds like a "no true Scotsman" statement. How are you defining "real"? Some human, somewhere, has to describe how to turn high-level language constructs into machine code. "When you see this pattern, emit this sequence of bytes." That's just templates and stencils. There's no magic for turning source code into machine code by divining the ISA at compile time. Anything that's taking source code and, at the time of execution, is compiling it to machine code on-the-fly is JIT compilation. Regardless how long it takes, lack of optimization, or which machine is the target (x86, ARM32/64, RISC-V, JVM, WebAssembly), it's JIT.
- atiedebee 2mo ago
  I thought the difference was that a JIT is used alongside an interpreter? It wouldn't need to do optimization, just compile while interpreting?
  - jagged-chisel 2mo ago
    How is “alongside an interpreter” meaningfully different than “at the time of execution”?
    - atiedebee 2mo ago
      "at the time of execution" could mean that your program never runs an interpreter and instead compiles the input code before running any of it, basically like the --run option in the tiny c compiler

The problem with the approach is that it's not real JIT-compilation, it's just assembly templates with basic substitutions. By not using LLVM, you're missing all the optimizations it does.

mort96 2mo ago

A non-optimizing compiler is a real compiler.

mgaunard 2mo ago
People do not call assemblers compilers, and yet, it's doing most of what the "JIT compiler" is doing.

IshKebab 2mo ago

This absolutely is real JIT compilation. Copy and patch is a very well known JIT compilation technique.

the-lazy-guy 2mo ago

This is absolutely a JIT-compiler. It compiles code into machine code. This is a surprisingly efficient way to get noticeable speedup relative to interpretation. Also it is much safer than proper optimising compiler. Say ebpf jit-compiler functions very similarly, because it is fast and _secure_ way to jit. (well, there's a bit of cheating because before emitting bpf bytecode it goes through gcc/clang pipeline). LLVM is a large dependency if you need to JIT. There are plenty of smaller (and much faster) alternatives which are much better fit for smaller projects. Larger projects usually roll out their own jit-pipeline because they can integrate better with the source language/interpreter and apply tricks LLVM is not well suited to (say, LLVM is not great at deoptimisation). I think only Julia is really a heavy user of LLVM JIT, also it is known for extremely slow repl from time to time.

mort96 2mo ago
To back up the "surprisingly efficient way to get a speed-up" thing: I once wrote a toy compiler, without an optimizer and without even a register allocator (so all variables lived in stack memory). In my test benchmarks it was roughly 4x slower than Clang at -O3, IIRC. That's not exactly blazing fast for a low level C-like language, but it's not bad. It's infinitely faster than what I've ever gotten a toy interpreter to be.

bastawhiz 2mo ago

By choosing LLVM you're also taking a serious latency hit, and potentially burning a ton of CPU cycles on optimizations that will never apply. JSC, for instance, implemented LLVM for FTLJIT (which is where many of the JS bits jangling around in LLVM originated from), but it was only useful for code that was highly likely to benefit from the optimizations because of the high cost of compilation. The webkit folks have since ripped out LLVM and replaced it with their own specialized JIT, which is essentially just what's demonstrated here (with some optimization passes).

RetroTechie 2mo ago

> By not using LLVM, you're missing all the optimizations it does And yet, a good portion of software that runs today's world is written in scripting languages & executed using interpreters. Which is okay! Imho: multiply [# of users] with [how often each user sees that software's effect] and [how much that contributes to the overall user experience], then you get a ballpark idea of how much $$/effort is worth spending on optimization. In other words: for a one-off, don't bother. But as usercount, frequency of use by individual users, poor UX or RAM/CPU consumption goes up, progress from script -> compiled -> optimizing compiler -> (if necessary) hand-optimized assembly as needed. And of course consider high-level design, data structures, algorithms etc in that process. A change there might be more effective than a switch from interpreted -> optimizing compiler. "Developer time" should not factor into that much (again: imho) unless users=developers. Thoughtlessly putting every change through a (slow?) pipeline that does 'random' toolbox-of-optimizations without need, is wasteful. Apply that toolbox as needed while keeping the above in mind.

jagged-chisel 2mo ago

Sounds like a "no true Scotsman" statement. How are you defining "real"? Some human, somewhere, has to describe how to turn high-level language constructs into machine code. "When you see this pattern, emit this sequence of bytes." That's just templates and stencils. There's no magic for turning source code into machine code by divining the ISA at compile time. Anything that's taking source code and, at the time of execution, is compiling it to machine code on-the-fly is JIT compilation. Regardless how long it takes, lack of optimization, or which machine is the target (x86, ARM32/64, RISC-V, JVM, WebAssembly), it's JIT.

atiedebee 2mo ago
I thought the difference was that a JIT is used alongside an interpreter? It wouldn't need to do optimization, just compile while interpreting?
- jagged-chisel 2mo ago
  How is “alongside an interpreter” meaningfully different than “at the time of execution”?
  - atiedebee 2mo ago
    "at the time of execution" could mean that your program never runs an interpreter and instead compiles the input code before running any of it, basically like the --run option in the tiny c compiler