5 ms·
JITs do more than just profile-guided optimizations. Their secret weapon is speculative optimizations that mean they don't need to work hard (and often fail) to
by pron 7y ago
JITs do more than just profile-guided optimizations. Their secret weapon is speculative optimizations that mean they don't need to work hard (and often fail) to prove the soundness of certain optimizations. They're allowed to guess and be wrong.
- the8472 7y agoAnother advantage they have is that they can focus the spent optimization cycles on hot code. AOT compilers can't afford running optimization passes in a loop (inline, optimize, inline, optimize, ...) until they reach a fixed point because that would blow up compile times if that were applied to the whole program.
- astrange 7y agoThis almost never matters. You just start at the leaves and go up and then you're done. Most people aren't interested in complicated superoptimizations, because a predictable compiler is more important.
- pron 7y agoThat's not true. A lot of very simple and useful optimizations are very hard to prove correct (e.g. devirtualization) and so can't be done with AOT compilers. It doesn't matter to people using languages that require the programmer to carefully control the compiler -- like C/C++ or Rust -- but it matters a great deal to languages that offer a smaller number of more general abstractions. It is virtually impossible to compile, say, JavaScript efficiently with an AOT compiler, but when compiled with a JIT it can have excellent performance.
- amelius 7y agoIf they do speculative optimizations, then doesn't that open the door to Spectre-like vulnerabilities, but now at the compiler level?
- chrisseaton 7y agoThis isn't speculation as in speculative execution, it's speculation as in speculating that a condition is true that cannot be proved to be true, so it's not the same thing. An example of this kind of speculation is speculating that there will only ever be one thread in a system, and removing locks. If that speculation ever proves to be wrong - a second thread is created - the locks are put back into the system. That doesn't related to spectre, as when the speculation is reversed the whole programs is first brought to a safe halt - it isn't fine-grained enough to be useful for Spectre. Unrelated, it is true that compilers need to be aware of Spectre-like vulnerabilities, and Graal does include experimental support for that.
- amelius 7y agoOk, but in a multi-user system (e.g. a webserver), if user 1 triggers a (de-)optimization, then user 2 can tell that a previous user was in that code path. Now I don't know how to extract useful information from that fact, but it shows that at least some information spills over the user boundaries.
- chrisseaton 7y agoOh I see what you mean - Spectre-like rather than specifically Spectre. Yes I suppose specialisation (more generally than speculation) could leak information, in the same way as cache status can leak information.
- pvg 7y agoWait, ignore soundness? How does that work/provide advantage?
- zlynx 7y agoA function could be compiled assuming the passed argument is always 2. All of the code for other values is just left out. As long as the compiled code has a check for values that are not 2 this code works great. It isn't correct though. At least, that's my interpretation.
- pvg 7y agoMaybe I'm getting tripped up in the terminology here but to me this case is still a JIT jittin' - you look at runtime data and decide it's worth it to crank out a special case optimization for the input of 2. You produce that that optimization, which is sound along with a check to make sure it is applied only in the special case. You get to defer other optimization. The advantage here still seems to come from the runtimeness of things rather than from being clever about soundness and there's really no guessing about soundness. So perhaps that's not it.
- chrisseaton 7y ago> to me this case is still a JIT jittin' I think the point is that some JITs never do this kind of optimisation - they just produce the same code an AOT compiler would, but at runtime. Such as the .NET JIT.
- pvg 7y agoI don't think that's the point the comment I'm replying to is making, or at least, it's not the point I'm asking about. Edit: Your example in the other comment about the locks is the sort of thing I'm asking about. There, an optimization is made which is sound under some specific conditions and then unmade when those conditions change.
- 7y ago