10 ms·
A Deep Introduction to JIT Compilers: JITs are not very Just-in-time
- jakozaur 6y agoTl;dr: JIT is a compiler that optimuzes certain parts of code after interpreter see it as hot (e.g. was executed many time). Thank to runtime information it can occasionally exceed performance of statically compiled language.
- kipply 6y agoexcept Julia because Julia <3
- dunefox 6y agoWhat do you mean?
- kipply 6y agoJulia compiles at runtime which makes it a JIT but it does not actually use runtime information to compile (except to know what to compile and maybe some other things) or have a concept of "hotness" (this is in the post~)
- MaxBarraclough 6y agoOr .Net, which entirely lacks an interpreter, and JIT compiles everything that it executes. (Like Julia, if I understand correctly.) JIT != mixed-mode execution.
- saagarjha 6y agoMono has one, AFAIK: https://www.mono-project.com/news/2017/11/13/mono-interpreter/ https://www.mono-project.com/news/2017/11/13/mono-interprete...
- chrisseaton 6y agoSome people make a distinction between 'just-in-time compilation' (like .NET - it's normal static compilation, but done just before first execution) and 'dynamic compilation' (like Java, it choses when to compile, uses dynamic information, may re-compile, etc.)
- ddragon 6y agoIn the Julia community they frequently call it Just Ahead-Of-Time the compiler strategy of making the inference of all downstream types and methods to dispatch and compiling all at once. And the period while purely running a static program "world age", which usually lasts until you go back to the global namespace or you need any dynamic feature. Julia does not have an interpreter outside of the debugger.
- dialamac 6y ago> Thank to runtime information it can occasionally exceed performance of statically compiled language. Interestingly in 30 years I have not once heard of a case where this theoretical benefit has manifested as a clear advantage in any real world application when looking at the system as a whole... amdahls law and all that. You can always hand tune the 1-10% hotspots for reasonable cost most of the time, and even static tools can do PGO which generally gets you where JIT would anyway.
- chrisseaton 6y ago> in 30 years I have not once heard of a case where this theoretical benefit has manifested as a clear advantage in any real world application Today you make make a very direct empirical comparison to see this - using the Graal compiler. This lets you compile exactly the same Java code either ahead-of-time or just-in-time, but using the same compiler logic except for the runtime information available when running just-in-time. The just-in-time code is (ignoring startup and warmup time) in my experience always faster, due to the extra runtime information.
- darksaints 6y agoHave you tried graal's PGO?
- chrisseaton 6y agoNo unfortunately - it's a closed-source Enterprise feature I believe. But the current differential is pretty large and nobody is shouting loud that they can fix it using the PGO that I've heard. And what will the PGO determine that a JIT can't also do?
- darksaints 6y agoA number of things actually. For one, most JIT implementations only optimize once, instead of continuously. The result is machine code that is optimized for the sorts of things done at startup, as opposed to steady state operation. For example, I have a Play app that takes a minute to start up. The JIT does a great job of optimizing the code that is called during the setup process, but the API code itself doesn't get much optimization. With PGO, I can get a more representative profiling dataset, allowing the JIT to see actual production loads instead of startup loads. And for CLI apps, PGO code starts up fast and never slows down to profile or optimize.
- glitchc 6y agoI don’t believe JITs can ever beat AOT compilation. However, if it can achieve parity in the execution of the critical path, that is sufficient to call it a win.
- Reelin 6y ago> I don’t believe JITs can ever beat AOT compilation. In certain cases [1] JIT can beat AOT by a decent margin because it has access to live information that isn't available at compile time. In theory you could AOT compile, profile, and recompile but in practice my understanding is that nothing beats doing it live. Consider that if the workload suddenly changes in production and your heuristic violates an invariant as a result, you can deoptimize, reprofile, and recompile everything on the fly. But wait, you say! Can't an AOT compiler do that? But no, the state space is almost certainly going to be too large by many orders of magnitude. So unless your AOT compiler effectively inserts a JIT compiler ... [1] Sorry, no examples to hand. Nobody knows but I'm actually a dog. Don't trust me!
- caballeto 6y agoDart VM is a good example. Their JIT is frequently faster than the AOT compiled code, although they may improve the AOT speed in future releases.
- pjmlp 6y agoYes they can in these scenarios: - Dynamic method calls that get de-virtualized, because there is only one visible implementation, including across shared libraries - Inlining across shared libraries - Adaptation of new CPU opcodes, for example a Java application written in 2000 could in theory make use of AVX, without being recompiled. - In the same vein, that is how shading languages work
- kipply 6y agoIt's also in the post, but https://carolchen.me/blog/jits-impls/ https://carolchen.me/blog/jits-impls/ has more advanced concepts
- cs702 6y agoVery nice. I just looked at the author's resume.[a] It appears she is still in, or only very recently graduated from, high school. Is that right? Impressive! [a] https://carolchen.me/ https://carolchen.me/
- kipply 6y agoYep, class of 2019, jit to actually have a physical prom and graduation
- cs702 6y agoI'm impressed! Others here are too, I'm sure. And lucky you. (Seniors this year got a raw deal.) PS. Great use of "jit" in a different context.
- scarmig 6y agoI see what you did there. Great blog, and you're into aerials! Always exciting to see someone else here who gets their invert on.
- brabel 6y agoHow does a high school kid get so interested in (and manages to pull off a great article about) stuff like Intermediate Representations, compiler optimisation techniques, JITs as different as those of Julia and GraalVM, that would easily scare even a seasoned programmer with years of experience and a CompSci degree under the belt?
- enriquto 6y agoYou don't know many nerds, do you? EDIT: When I was in high school, msdos and windows viruses were all the rage: self-modifying polymorphic code that injected itself into the RAM of other processes, having no identical strings longer than a few bytes between each instance of the virus, etc. Way useless than jit stuff, but a comparable level of "depth", and 100% assembly code.
- coolreader18 6y ago
- saagarjha 6y agoI just realized that the link color changes every time I load the page–I clicked on the article again and it was different from the last time I looked at it!
- pjmlp 6y agoInteresting articles, my only remark is not talking about JIT caches and PGO across process executions, but I guess it might come on a later post.
- kipply 6y agohttps://carolchen.me/blog/jits-intro/#pogo https://carolchen.me/blog/jits-intro/#pogo It's there though not very in depth c:
- pjmlp 6y agoThis is not what I am talking about, JIT caches with PGO data work in a different way. https://openjdk.java.net/jeps/310 https://openjdk.java.net/jeps/310 https://www.eclipse.org/openj9/docs/xcodecachetotal/ https://www.eclipse.org/openj9/docs/xcodecachetotal/ https://docs.oracle.com/cd/E13188_01/jrockit/docs142/userguide/codecach.html https://docs.oracle.com/cd/E13188_01/jrockit/docs142/usergui... https://source.android.com/devices/tech/dalvik/jit-compiler#architectural-overview https://source.android.com/devices/tech/dalvik/jit-compiler#...
- kipply 6y agoI don't think I'm familiar with this (I may have come across it but didn't identify it by name when I worked with Graal?), could you provide a link? :o Thanks!
- pjmlp 6y agoI have provided links above, there are also a couple of Java Language Summit, Code/Java ONE and Google IO talks about them, but I would need to search for them. Can provide them later.
- kipply 6y agoThanks, I know of this stuff but I don't think I would've been qualified to form an explanation/overview of them. It would be nice to have these in a post though!
- joe_the_user 6y agoThis article is really nice for not making things harder than they needed to be. Trace then substitute the trace - no magic. It seems like a big part of all these strategies is making sure the checks for the optimizations (checking if they're possible, checking if they're done) don't take more time than the optimizations save. Or could there be a way to alter the execution graph so that once you add in the optimization, you never have to check if it's there?
- the8472 6y ago> It seems like a big part of all these strategies is making sure the checks for the optimizations (checking if they're possible, checking if they're done) don't take more time than the optimizations save. You can't really check that, you can only hope your heuristics are right. You don't know in advance how many times a method will be executed and thus how much optimization budget is worth it. If you boot up a java application that runs for months and handles tons of traffic then you could theoretically justify many minutes of CPU cycles for optimization. And that's not all that makes JITs complex. Some also do speculative optimizations that require guards and deoptimization points so they can fallback and recompile if those assumptions are violated. If the language has a garbage collector then the GC and JITs have to cooperate for ideal performance.
- Leszek 6y ago> Or could there be a way to alter the execution graph so that once you add in the optimization, you never have to check if it's there? One of the things V8 does is store "field dependencies" -- a list of optimized code attached to fields that is invalidated if that field is modified in a certain way. The optimized code is known to only access that field in a certain way, thus doesn't need to check it, so this only slows down unoptimized paths (and only on their slow paths too, since even unoptimized code has fast paths for known access patterns thanks to inline caches). Comes with a memory/generality cost though, so this isn't used for every type of check.
- vlovich123 6y agoIs anyone aware of any efforts to get clang/rustc to use lli to speed up iteration times? My thought is you lower everything to llvm bytecode first but skip all the codegen/linking. Then you can codegen or run interpreted. Might help make debug builds much faster to get to execution. Probably could trivially parallelize the background compilation & auto-build in the background on every file change to get the best of all worlds (most code gen is ready, the remainder is executable & will get compiled as needed). If you can do it with live substitution (which I think is what lli does) then you'd progressively get a faster program.
- saagarjha 6y agoHow much time actually goes into instruction selection and register allocation? I thought a lot of the time went into optimization passes at the IR level.
- jkubrynski 6y agoThere is also an interesting presentation about how Azul implemented their JIT called Falcon, that is fully based on LLVM. https://www.youtube.com/watch?v=Uqch1rjPls8 https://www.youtube.com/watch?v=Uqch1rjPls8 Generally LLVM is a nice idea that allows vendors to reuse major components.
- saagarjha 6y agoThe one problem with LLVM is that it’s not very well suited to JIT compilers, a major component of which is its slowness.
- why_only_15 6y agoThis is a nit but looks like she says pyc files aren't around in python3 anymore -- they're still there just in the __pycache__ directory (as of 3.2)
- saagarjha 6y agoI know Python does the dunders and all, but couldn’t they have at least stuck a dot in front of it if they wanted to move it out of the way :(
- Cu3PO42 6y agoEntirely off-topic, but the author's name when written without a space (i.e. Carolchen, as it is in the URL) just happens to be the diminutive of Carol in German. Except that Carol is not a very common name here at all, so rare in fact that I've not once met a German Carol. This had me very confused for a moment.
- wikunia 6y agoFirst of all! Nice post :) Just some random things: The first paragraph needs some commas, I think. In the first code block you probably want to have `print(variable_x)`