4 ms·
If JIT-ing a statically compiled input makes it faster, does that mean that JIT-ing itself is superior or does it mean that the static compiler isn't outputting
by freedomben 4y ago
If JIT-ing a statically compiled input makes it faster, does that mean that JIT-ing itself is superior or does it mean that the static compiler isn't outputting optimal code? (real question. asked another way, does JIT have optimizations it can make that a static compiler can't?)
- vips7L 4y agoYes, the JIT has more profile guided data as to what your program actually does at runtime, therefore it can optimize better.
- gpderetta 4y agoOn the other hand some optimization are so expensive that a JIT just doesn't have the execution budget to perform them. Probably the optimal system is an hybrid iterative JIT/AOT compiler (which incidentally was the original objective of LLVM).
- andrewaylett 4y agoIt depends on what the JIT does exactly, but in general yes a JIT may be able to make optimisations that a static compiler won't be aware of because a JIT can optimise for the specific data being processed. That said, a sufficiently advanced CPU could also make those optimisations on "static" code. That was one of the things Transmeta had been aiming towards, I think.
- kmeisthax 4y agoIt's more the case that the ahead-of-time compilation is suboptimal. Modern compilers have a thing called PGO (Profile Guided Optimization) that lets you take a compiled application, run it and generate an execution profile for it, and then compile the application again using information from the profiling step. The reason why this works is that lots of optimization involves time-space tradeoffs that only make sense to do if the code is frequently called. JIT only runs on frequently-called code, so it has the advantage of runtime profiling information, while ahead-of-time (AOT) compilers have to make educated guesses about what loops are the most hot. PGO closes that gap. Theoretically, a JIT could produce binary code hyper-tailored to a particular user's habits and their computer's specific hardware. However, I'm not sure if that has that much of a benefit versus PGO AOT.
- com2kid 4y ago> Theoretically, a JIT could produce binary code hyper-tailored to a particular user's habits and their computer's specific hardware. However, I'm not sure if that has that much of a benefit versus PGO AOT. In theory JIT can be a lot more efficient, optimizing for not only the exact instruction set, and do per CPU architecture optimizations, such as instruction length, pipeline depth, cache sizes, etc. In reality I doubt most compiler or JIT development teams have the resources to write and test all those potential optimizations, especially as new CPUs are coming out all the time, and each set of optimizations is another set of tests that has to be maintained.
- saagarjha 4y agoThe well funded production JIT compilers (HotSpot, V8, etc.) absolutely do take advantage of these. The vector ISA can sometimes be unwieldy to work with but things like replacing atomics, using unaligned loads, or taking advantage of differing pointer representations is common.
- com2kid 4y agoThey do some auto-vectorization, but AFAIK they don't do micro-optimizations for different CPUs.
- duped 4y agoLike another commented, JIT compilers do this today. The thing that makes this mostly theoretical is that the underlying assumption is only true when you neglect that an AOT has zero run-time cost while a JIT compiler has to execute the code it's optimizing and the code to decide if it's worth optimizing and generate new code. So JIT compiler optimizations are a bit different than AOT optimizations since they have to both generate faster/smaller code and the execute code that performs the optimization. The problem is that most optimizations beyond peephole are quite expensive. There's another thing that AOT compilers don't need to deal with, which is being wrong.Production JITs have to implement dynamic de-optimization in the case that an optimization was built on a bad assumption. That's why JITs are only faster in theory (today), since there are performance pitfalls in the JIT itself.
- mockery 4y agoIn addition to the sibling comments, one simple opportunity available to a JIT and not AOT is 100% confidence about the target hardware and its capabilities. For example AOT compilation often has to account for the possibility that the target machine might not have certain instructions - like SSE/AVX vector ops, and emit both SSE and non-SSE versions of a codepath with, say, a branch to pick the appropriate one dynamically. Whereas a JIT knows what hardware it's running on - it doesn't have to worry about any other CPUs.
- acdha 4y agoOne great example of this was back in the P4 era where Intel hit higher clock speeds at the expense of much higher latency. If you made a binary for just that processor a smart compiler could use the usual tricks to hit very good performance, but that came at the expense of other processors and/or compatibility (one appeal to the AMD Athlon & especially Opteron was that you could just run the same binary faster without caring about any of that[1]). A smart JIT could smooth that considerably but at the time the memory & time constraints were a challenge. 1. The usual caveats about benchmarking what you care about apply, of course. The mix of webish things I worked on and scientists I supported followed this pattern, YMMV.
- duped 4y agoAOT compilers support this through a technique called function multi-versioning. It's not free and only goes so far, but it isn't reserved to JITs. The classical reason to use FMV is for SIMD optimizations, fwiw
- ketralnis 4y agoIt means that in this case, the static compiler emitted code that could be further optimised, that's all. It doesn't mean that that's always the case, or that static compilers can't produce optimal code, or that either technique is "better" than the other. An easy example is code compiled for 386 running on a 586. The A->A compiler can use CPU features that weren't available to the 386. As with PGO you have branch prediction information that's not available to the static compiler. You can statically compile the dynamically linked dependencies, allowing inlining that wasn't previously available. On the other hand you have to do all of that. That takes warmup time just like a JIT. I think the road to enlightenment is letting go of phrasing like "is superior". There are lots of upsides and downsides to pretty much every technique.
- rowanG077 4y agoA JIT can definitely make optimizations that a static compiler can't. Simply by virtue of it having concrete dynamic real-time information.