5 ms·

> JIT compilers are a thing, so this statement is just plain false. Do you know of any JITed languages that approach SIMD optimized native code in terms of per

> JIT compilers are a thing, so this statement is just plain false. Do you know of any JITed languages that approach SIMD optimized native code in terms of performance and overhead? Even the best ones (eg. LuaJIT) fail to come close to even moderately optimized C code, with much higher memory usage. On top of that, JIT optimizations don't always trigger and you fall back to interpreted levels, which are orders of magnitude more expensive.

kaba0 3y ago
Why would LuaJIT be the best? I’m fairly sure that V8 and HotSpot that had million times more dev time spent on are way better on every front. And autovectorization is definitely done by HotSpot (I would guess also by V8, though not sure), but autovectorization in itself is very finicky both AOT and JIT, so there is now a stronger move towards user-exposed controls, e.g. Java’s new Vector API, but C# also had something like that for years. Funnily enough, C doesn’t have any standard way to control vectorization, only compiler-specific extensions. Also, memory layout is a completely different problem space, don’t judge JIT compilers on that. On numerical code they are very competitive with AOT compiled, low-level languages, both JavaScript and Java. Of course if a given object could not be escape analyzed than the semantics doesn’t allow further inlining and somewhere that pointer indirection have to be paid, but don’t forget that besides benchmarks, most real life code in C/C++/Rust whatever will also happily chase pointers, so the difference is again, not as stark as some benchmark may tell you. Also, this is more related to GC+JITted languages, but are not necessary for JIT alone. With that said, I just realized that there is Julia in this exact niche, that also demonstrates my point very well. > On top of that, JIT optimizations don't always trigger and you fall back to interpreted levels, which are orders of magnitude more expensive. An optimization “triggering” is exactly similar to what an AOT compiler would do, only with a smaller time/resource budget. What you are getting at - speculative optimizations - are a different story, and are not a necessity in case of a JIT. And they get deoptimized when the assumptions they made for the optimization are proven incorrect (e.g. in java’s case virtual calls may have been replaced by static ones as only a single class implementing the given interface were loaded at the time - but upon loading a new implementation this will have to be deoptimized). You are right that deoptimizations can be expensive, when they go wrong, or in the worst case go into some cycles, but in practice with V8/HotSpot, these work well enough. The initial warmup is more likely to cause some jitter if any.
- fredrikholm 3y ago
  Thank you for your answer, interesting read! > Why would LuaJIT be the best? It was one example out the group of best ones, sorry for not being clear enough. > Also, memory layout is a completely different problem space, don’t judge JIT compilers on that. Exactly; you're stuck with the memory floor the language provides you with, JIT or not. This means a fixed minimum cost (resources, heat, electricity etc) that would otherwise not be there. > On numerical code they are very competitive with AOT compiled, low-level languages How competitive? I've never seen a benchmark show a JITed language compare to the equivalent program in C, with the few exceptions where the C implementation was written in such a way as to enable the author to say "we're faster than C". > most real life code in C/C++/Rust whatever will also happily chase pointers, so the difference is again, not as stark as some benchmark may tell you Oh most definitively. On the other side of that same coin, most real life code in JS/Java/C#/PHP/Ruby consume orders of magnitude more resource than their lower level counter parts. Often, a lot of that is due to writing idiomatic code in languages where idioms are expensive (eg. `[... new Set(arr)]` in JS), which often also means that underlying libraries and frameworks also use these idioms. Even worse when libraries like lodash/ramda become popular and you have another order of magnitude more overhead compared to the equivalent for loops. You could of course optimize that, just like you could optimize your C/C++/Rust/Zig program. > You are right that deoptimizations can be expensive, when they go wrong, or in the worst case go into some cycles If what you're saying is true than that's great news! My own experiences (>decade ago) of observing other teams experiment with JITed languages showed more pessimistic worst case outcomes (embedded engineers trying out embedded scripting languages).
  - xgdgsc 3y ago
    I' ve seen quite a few posts like https://users.rust-lang.org/t/a-performance-problem-compared-with-julia/51871 https://users.rust-lang.org/t/a-performance-problem-compared... , https://discourse.julialang.org/t/why-is-julia-faster-than-c-for-quicksort/102669 https://discourse.julialang.org/t/why-is-julia-faster-than-c... ... And many are from average programmers writing Cpp/Rust vs Julia, this is really good news for scientists/engineerers who need quick experiments.
  - 3y ago
pjmlp 3y ago
Java and .NET, because you can use SIMD primitives directly.
jasonwatkinspdx 3y ago
Julia. In fact delaying code generation as late as possible in that language is key to SIMD efficiency because the JIT can re-order loops, etc, with the knowledge of runtime values.

kaba0 3y ago

Why would LuaJIT be the best? I’m fairly sure that V8 and HotSpot that had million times more dev time spent on are way better on every front. And autovectorization is definitely done by HotSpot (I would guess also by V8, though not sure), but autovectorization in itself is very finicky both AOT and JIT, so there is now a stronger move towards user-exposed controls, e.g. Java’s new Vector API, but C# also had something like that for years. Funnily enough, C doesn’t have any standard way to control vectorization, only compiler-specific extensions. Also, memory layout is a completely different problem space, don’t judge JIT compilers on that. On numerical code they are very competitive with AOT compiled, low-level languages, both JavaScript and Java. Of course if a given object could not be escape analyzed than the semantics doesn’t allow further inlining and somewhere that pointer indirection have to be paid, but don’t forget that besides benchmarks, most real life code in C/C++/Rust whatever will also happily chase pointers, so the difference is again, not as stark as some benchmark may tell you. Also, this is more related to GC+JITted languages, but are not necessary for JIT alone. With that said, I just realized that there is Julia in this exact niche, that also demonstrates my point very well. > On top of that, JIT optimizations don't always trigger and you fall back to interpreted levels, which are orders of magnitude more expensive. An optimization “triggering” is exactly similar to what an AOT compiler would do, only with a smaller time/resource budget. What you are getting at - speculative optimizations - are a different story, and are not a necessity in case of a JIT. And they get deoptimized when the assumptions they made for the optimization are proven incorrect (e.g. in java’s case virtual calls may have been replaced by static ones as only a single class implementing the given interface were loaded at the time - but upon loading a new implementation this will have to be deoptimized). You are right that deoptimizations can be expensive, when they go wrong, or in the worst case go into some cycles, but in practice with V8/HotSpot, these work well enough. The initial warmup is more likely to cause some jitter if any.

fredrikholm 3y ago
Thank you for your answer, interesting read! > Why would LuaJIT be the best? It was one example out the group of best ones, sorry for not being clear enough. > Also, memory layout is a completely different problem space, don’t judge JIT compilers on that. Exactly; you're stuck with the memory floor the language provides you with, JIT or not. This means a fixed minimum cost (resources, heat, electricity etc) that would otherwise not be there. > On numerical code they are very competitive with AOT compiled, low-level languages How competitive? I've never seen a benchmark show a JITed language compare to the equivalent program in C, with the few exceptions where the C implementation was written in such a way as to enable the author to say "we're faster than C". > most real life code in C/C++/Rust whatever will also happily chase pointers, so the difference is again, not as stark as some benchmark may tell you Oh most definitively. On the other side of that same coin, most real life code in JS/Java/C#/PHP/Ruby consume orders of magnitude more resource than their lower level counter parts. Often, a lot of that is due to writing idiomatic code in languages where idioms are expensive (eg. `[... new Set(arr)]` in JS), which often also means that underlying libraries and frameworks also use these idioms. Even worse when libraries like lodash/ramda become popular and you have another order of magnitude more overhead compared to the equivalent for loops. You could of course optimize that, just like you could optimize your C/C++/Rust/Zig program. > You are right that deoptimizations can be expensive, when they go wrong, or in the worst case go into some cycles If what you're saying is true than that's great news! My own experiences (>decade ago) of observing other teams experiment with JITed languages showed more pessimistic worst case outcomes (embedded engineers trying out embedded scripting languages).
- xgdgsc 3y ago
  I' ve seen quite a few posts like https://users.rust-lang.org/t/a-performance-problem-compared-with-julia/51871 https://users.rust-lang.org/t/a-performance-problem-compared... , https://discourse.julialang.org/t/why-is-julia-faster-than-c-for-quicksort/102669 https://discourse.julialang.org/t/why-is-julia-faster-than-c... ... And many are from average programmers writing Cpp/Rust vs Julia, this is really good news for scientists/engineerers who need quick experiments.
- 3y ago

pjmlp 3y ago

Java and .NET, because you can use SIMD primitives directly.

jasonwatkinspdx 3y ago

Julia. In fact delaying code generation as late as possible in that language is key to SIMD efficiency because the JIT can re-order loops, etc, with the knowledge of runtime values.