12 ms·
Jitterdämmerung
- smcl 11y agoFor anyone confused at the title - it's a reference to a part of the epic Wagner opera "Der ring des nibelungen" called "Götterdämmerung" or "Twilight of the Gods"
- dmichulke 11y agoWhere "Dämmerung" is actually meant as the "twilight at sunset", a metaphor for when something comes to an end. Hence X-Dämmerung is "the end of X" for X = Gods|JIT in the above examples.
- cperciva 11y agoThe title is still confusing though: I immediately parsed "Jitterdämmerung" as "the twilight of jitter". Which almost made sense, given that JITs increase the variance of run times...
- coolandsmartrr 11y agoWhile as a Wagnerian I noticed this immediately, I feel the subject of this post was not exactly the scale to warrant an allusion to an epic[1] tetralogy. But then again, I could be being pedantic. [1] Frankly, no words would do justice to describe the magnitude of Wagner’s historical opus.
- cperciva 11y agono words would do justice to describe the magnitude of Wagner’s historical opus Perhaps "Lord of the Rings, for adults"? ;-)
- Intermernet 11y agoI personally rate it as one of the best titles for a blog post I've come across in a while :-) If someone eventually advances the state of distributed revision control systems then they'll get to write an article titled "Gitterdämmerung".
- tempodox 11y ago+1. Actually, the awaited advancement is here already (hg instead of git), but for the accolytes' adherence to the buzz. The cargo cult of `SCM == git` seems at the height of its power.
- gjm11 11y agoI think "Jitterdämmerung" is strictly better than "Gitterdämmerung" would be, because there really are multiple JITs but there's only one Git. (On the other hand, the Hamming distance to "Götterdämmerung" would be less.)
- Intermernet 11y agoIf we all took Dijkstra's advice we could claim that the current state of programming is "Gotodämmerung". Not so good on the Hamming distance though...
- tempodox 11y ago> "Twilight of the Gods" Taking all context into account, it's the “dusk” variety of twilight.
- Kristine1975 11y agoOne aspect the author doesn't talk about is emulation: The Wii/GameCube emulator Dolphin uses JIT to translate the console's PPC machine code to x86 machine code. I'm also not sure if "scripting languages" truly don't need a JIT. The development of LuaJIT for example has been sponsored by various companies, so there seems to be a need for the fast execution of Lua code: http://luajit.org/sponsors.html http://luajit.org/sponsors.html
- w0utert 11y agoYes, but to be fair, Lua is one of these few examples where JIT makes a lot of sense because the extreme simplicity of the language, which greatly expands the opportunities for a JIT to generate efficient code with relatively low overhead and JIT compiler complexity. This approach is great for some things, but it does sacrifice the flexibility and expressiveness of the language. Anything not part of the core language (which is a lot compared to most other programming languages, e.g. anything related to OOP, or more advanced data types than strings, floats and tables) has to be re-invented and/or bolted-on, which IMO makes it unsuitable for most kinds of applications. This should not be interpreted as critique about Lua the language bu the way, I'm a big fan of Lua for embedded scripting and I generally love tools with a narrow focus (as opposed to kitchen sink technology). I would not choose Lua for anything besides embedded scripting though.
- outworlder 11y agoWhat would you choose instead?
- w0utert 11y agoThat all depends on the application, I guess ;-) Usually I end-up using Python for smaller things that aren't mission-critical or need maximum performance, C++ for almost everything else, and Objective-C for OS X/iOS stuff. Maybe Java for things where both performance, safety and ease-of-development/maintenance matter (the latter would be mostly for other people who would have to work on the same projection who are less skilled in C++, as I'm not at all a fan of the language myself, but I recognize it has some properties that make it a suitable choice for many kinds of applications ;-). I don't have enough experience with any other programming languages that can be deployed easily across Linux and OS X, so I can't comment on those. Rust seems to have some good ideas so I may want to learn more about it in the future. If I wanted to write something for Windows platforms I'd most likely gravitate towards C#, from what I know about the language it appears to have all the good things of Java without its downsides.
- Animats 11y agoWhy did Java stay with a JIT for so long, anyway? Java is hard-compilable, and GCC can compile Java. The original reason for a JIT was to speed up applets running in the browser. (Anyone remember those?) Server side, there was never much point.
- Kristine1975 11y agoCynical answer: It wouldn't fit the "write once, run anywhere" slogan used to market Java.
- jerven 11y agoJIT gives you more information about how the program is actually run. This allows a large number of optimistic optimizations that GCC etc... can not make. This in theory allows faster code to be generated. Than ahead of time compilers can. In practice this is destroyed by the pointer chasing default datastructure in java. If you are executing "C" using JVM techniques then a research JVM JITing C code gets within 15% of GCC on a reasonable set of benchmarks. (http://chrisseaton.com/plas15/safec.pdf http://chrisseaton.com/plas15/safec.pdf). These benchmarks are favorable for the AOT GCC as they have few function pointers, startup configuration switches and little dataflow variety. I suspect that specific compiler optimizations on the C code for the dense matrix transform in GCC are bigger part of why GCC is faster than the fact that it is AOT instead of JIT. There are also a number of AOT compilers for Java e.g. excelsior. But I have not seen superior performance with that over an equivalent hotspot release.
- masklinn 11y ago> JIT gives you more information about how the program is actually run. This allows a large number of optimistic optimizations that GCC etc... can not make. They'd be surprised to learn that considering GCC supports profile-guided optimisation.
- Kristine1975 11y agoHowever a JIT can actually optimize the code for the current usage profile, whereas profile-guided optimizations in AOT compilation cannot.
- zamalek 11y agoThe problem with native code is platform disparity. It's absurd that so many binary variants are required when shipping code to e.g. Android or iPhone. This can apply to x86, too. Not sure if all your users have TSX-capable CPUs? You're now shipping two binaries. Not sure if your users have AVX? Now you have at least three binaries. This is the biggest advantage of intermediate languages - your code can at the very least execute everywhere with a single image, and in some cases automatically take advantage of platform features such as AVX. I think ART takes the correct approach: ship IL and AOT it on the device. Hopefully some day we can get the same type of system for LLVM IR.
- Kristine1975 11y ago>This can apply to x86, too. The Intel C/C++ compiler creates different code paths depending on the CPU's capabilities. So at least there's no duplication of non-cpu-specific code. It also does or at least used to check for the CPU's capabilities by using the CPU's vendor ID string, resulting in AMD CPUs executing the slowest code path: http://agner.org/optimize/blog/read.php?i=49#49 http://agner.org/optimize/blog/read.php?i=49#49 (Intel got sued for this by AMD: https://en.wikipedia.org/wiki/Intel_C%2B%2B_Compiler#Criticism https://en.wikipedia.org/wiki/Intel_C%2B%2B_Compiler#Critici...) *Edit: >Hopefully some day we can get the same type of system for LLVM IR. Right now the LLVM IR isn't platform-agnostic, e.g. when it comes to calling conventions. Also when Apple transitioned from 68000 to PPC in the 90ies, then from PPC to x86 and recently from 32 bit to 64 bit, they used "fat binaries" where a single executable file contained code for several platforms. That way one and the same application could run on old Macs with PPC, newer ones with 32 bit x86 CPUs and really new ones with 64 bit capable x86 CPUs.
- zamalek 11y ago> The Intel C/C++ compiler creates different code paths depending on the CPU's capabilities. Wow. That's a really slick way to solve the problem. Edit: > Right now the LLVM IR isn't platform-agnostic, e.g. when it comes to calling conventions. That's unfortunate. Either way I'd love to see some form of native IL that can be shipped instead of binaries.
- ilaksh 11y agoThis is one reason I am pretty excited about the potential for web assembly in terms of a cross-platform and cross-language performant target.
- pron 11y agoThere is no "shift away from JIT", just a simple observation that JITs are a tradeoff that you sometimes don't want to make. They have three drawbacks and two advantages. The three drawbacks are 1/ slow warmup, 2/ somewhat increased RAM and CPU usage and 3/ complex implementation. The two advantages are 1/ (often much) better runtime performance (i.e. more optimized code) and 2/ much better support for runtime manipulation of code (for profiling, debugging "at-full-speed", hot-patching, monitoring etc.). The slow warmup makes JITs a bad choice for quick command-line tools, and the increased RAM/CPU makes them a bad choice for battery-powered devices and those with very limited RAM. The better performance and optimization opportunities makes them the only performant choice for some languages that are hard to optimize AOT (those that rely a lot on dynamic dispatch and/or have dynamic data-structures, i.e. maps instead of class instances). The better instrumentation support makes them a terrific choice for long-running server-side application, where the drawbacks don't matter. On the client-side, unless the language requires a JIT for decent performance (like JS), there is no compelling reason to use one, and that's why Microsoft's decision makes perfect sense, as they've decided to focus on .NET on the client. This has nothing to do with JITs' great utility in general. One of the biggest breakthroughs in compiler technology in the last decade is Oracle Lab's Graal[1], which can also be used as an AOT, but with less-powerful optimizations. E.g. Graal does this: https://twitter.com/ChrisGSeaton/status/619885182104043520 https://twitter.com/ChrisGSeaton/status/619885182104043520 Graal (alongside its language-construction DSL, Truffle) has yielded implementations of Ruby, Python and JS that easily rival the state-of-the-art with far, far less effort, and also a very decent implementation of C (also with orders-of-magnitude less effort than the competition). The third drawback makes developing JITs from scratch a bad choice for anyone but the most well-resourced teams, or those that are in no hurry, or those that target very simple languages only. JITs have another drawback -- less predictable performance -- that matters mostly for hard-realtime code (or nearly hard-realtime), as deopts that momentarily slow-down performance as a better optimization strategy is sought. This is why hard-realtime JVMs offer a mixed JIT/AOT mode, where the hard realtime kernel (which values predictability over perfroamce) is AOT compiled, and the soft-realtime or non-realtime support code is JITted (which you want to run as fast as possible, but don't mind a rare hiccup). [1]: https://wiki.openjdk.java.net/display/Graal/Publications+and+Presentations https://wiki.openjdk.java.net/display/Graal/Publications+and...
- pvdebbe 11y agoI'm surprised to read about this direction, however little practiced it may be. JIT never was for the fastest of applications but in-time optimization should (and has, in some occasions) reach and even exceed the speeds of precompiled ('native'). Simply because the JIT optimizer has access to the actual runtime data that is being fed to the Tight Loop of Performance Bottleneck, and can do local and specific optimization for that run alone. AOT optimized binary can only do generic optimizations in theory.
- joosters 11y agoSeveral years ago, HP (I think) published a research paper investigating JIT on precompiled binaries, exactly for this purpose. They claimed around 5-10% speedup on x86 code. Unfortunately the project never got released.
- lmm 11y agoJIT may be a bad fit for real-time phone apps. But at the same time we're seeing the return of batch processes thanks to "big data". At my last job I worked on Spark processes that took several hours to run. In research, performance is important but you do get to average. So I don't think JIT will go away for that case; it wasn't worth hand-optimising (most jobs were only ran a couple of times), but at the same time I was very glad it wasn't Python.
- JohnDoe365 11y agoWith Big Data we also see stream processing, which is much more Big Data than batch processing
- pjmlp 11y agoFully agree, JIT makes sense in dynamic languages or real-time conversion of op-codes, for everything else AOT is much better solution. Having already quite a good experience with memory safe languages when Java came into the world (mostly Wirth and ML derived languages), I never understood why Sun went with the JIT approach. Other than trying to sell Java to run in the browser. There were already quite a few solutions using bytecodes as portable executable format that were AOT compiled at installation time, e.g. mainframes and research OSes like Native Oberon variants. The best is to have both on the toolchain. A interpreter/JIT for development and REPL workflows. Leaving the AOT for deployment/release code.
- pron 11y ago> for everything else AOT is much better solution Not if everything else includes long-running servers that may require various kinds of fiddling at runtime, like injection or turning off and on of monitoring/troubleshooting code, nor if you want to have zero-cost abstractions that are actually high-level abstractions (as opposed to the other kind of zero-cost abstractions, that are basically just weak abstractions that can be offered for free).
- pjmlp 11y agoOnce upon a time I did drank too much of JIT Kool-Aid, and for a moment I though it was the future, kind of. However bearing the scars of the pain of "kinds of fiddling at runtime, like injection or turning off and on of monitoring/troubleshooting code" actually means in practice, changed my mind. It is so damn hard to tune them, that there are consulting companies that specialize in selling JIT tunning services. And in the end they offer little performance advantage over using PGO or writing code that makes better use of value types and cache friendlier algorithms. This was one reason why Microsoft introduced PGO for .NET in .NET 4.5. So yes, I really mean everything else.
- pron 11y agoI've gone the opposite way, and I really like my long-running server code to be JITted when possible. I only wish my old C++ servers could be tweakable at runtime as my Java servers[1] and I really like being able to use languages (even for DSLs or rule engines) that have a few powerful abstractions, and let the JIT make them run nearly as fast as the infrastructure code. If you really need careful tuning of optimizations (which is never a walk in the park), it will be made much nicer with this: http://openjdk.java.net/jeps/165 http://openjdk.java.net/jeps/165 [1]: Just like Erlang, only fast! ;)
- pete23 11y agoStatic typing in Java does not give enough information to optimise because of polymorphism. The JIT can observe runtime behaviour and inline method calls at monomorphic sites. http://insightfullogic.com/2014/May/12/fast-and-megamorphic-what-influences-method-invoca/ http://insightfullogic.com/2014/May/12/fast-and-megamorphic-... Worth noting also that Azul's Zing ReadyNow technology for AOT (and indeed their no-pause C4 collector) address some of the specific gripes raised here around the JVM...
- fmstephe 11y agoZing's ReadyNow is useful but really highlights some of the problems with JITs (in some domains). I think that ReadyNow is very interesting because its existence points to some interesting consequences of JITs in performance sensitive domains. The driving force behind ReadyNow (as I understand it) was that many performance sensitive systems needed to be fast out of the gates on start up. This means that an interpreted->compiled->optimised-compiled transition was not acceptable. Developers would try to solve this by running dummy data through the system before it was open to real traffic. But this had the unfortunate consequence that the JIT would optimise for dummy data only, including clever inlining at, apparently, monomorphic call sites etc. When real traffic flows into the system the JVM sees that its optimisations are no longer effective/valid. This would trigger a de-compilation/re-compilation of many code sites causing a very noticeable stutter. Now we have ReadyNow. If you are really committed to Java or the JVM and you don't like these JIT stutters this is your solution. But this is an extra layer of complexity and another thing to be managed and another thing to fail. This is on top of your JVM, jar file soup you may already be struggling with. I would prefer a good AOT compiler to remove this concern and give me quite good predictable performance. YMMV of course. Sources http://www.azul.com/products/zing/readynow-technology-for-zing/ http://www.azul.com/products/zing/readynow-technology-for-zi... https://groups.google.com/forum/#!searchin/mechanical-sympathy/ReadyNow/mechanical-sympathy/QT_P5dGXjXw/W9uiT-dSqjMJ https://groups.google.com/forum/#!searchin/mechanical-sympat...
- jdmichal 11y agoThis is absolutely correct. The primary optimization in Java JIT is determining the concrete types (potentially) present at the call site and thereby turning dynamic method dispatch (vtable) into static method dispatch. Static method dispatch then allows inlining.
- yummyfajitas 11y agoThe article seems to ignore a variety of very real benefits that you get from the JIT. In many real world cases, there are optimizations that the JIT can perform. For example: final boolean runF = ...; //In *this* program run, it works out to be false. ... if (runF) { f(x); } The JIT can just eliminate this branch entirely. More interestingly: if (cond) { myVar = f(x); //f(x) has provably no side effects } Supposing cond is nearly always true, the JIT can optimistically run f and in parallel check cond. In the unlikely event that cond is false, the JIT will still compute the return value of f(x), but just not do the assignment. It surprised me when I saw all this happen, but the JVM's JIT is actually pretty smart and works very well for long running server processes.
- xyzzyz 11y agoIn the first case, either the "if" is run in a tight loop, and then the branch predictor will likely predict it correcly, or it isn't, and then branch elimination brings only trivial improvement. In the second, what do you mean by "run f and in parallel check cond"? How that would translate to the machine code? You want to run and synchronize cond and f(x) between two separate CPU cores or what?
- yummyfajitas 11y agoAnd if it's not run in a tight loop, the JIT can often still detect it. I don't know why you believe such things are trivial. On one occasion I altered some code which broke this, and it added about 10-20 seconds to a 15 minute HFT simulation. If that code made it to prod, those 10-20 seconds would be distributed around the hot parts of an HFT system. That's not remotely trivial, at least for performance critical applications (HFT, RTB, etc). In the second example, basically what happens (if you get lucky - I haven't found the JVM JIT to be super consistent) is that computation of cond and f(x) will be put into the pipeline together, and only the assignment to myVar will be conditional. This is strictly NOT something the CPU can do by itself, since there are side effects at the machine level (allocating objects, and then GCing them).
- tobias3 11y ago
- tempodox 11y agoThe Julia language seems to be designed to be nothing more than optimised glue between two Python scripts. As a dynamic language, it's certainly a good match for a JIT (run at once after a quick edit). And that also explains why there can never be a standalone executable produced from Julia, since that would require AOT. My guess is that Julia will dodge the Jitterdämmerung for a long time to come, if not forever.
- Fede_V 11y agoJulia does very little tracing, except to infer types, which it then passes on to LLVM. All the optimizations in Julia are because Julia stresses type stability, and once types are known, LLVM can generate very good code specific for those types. You can easily annotate functions with types in Julia, and pre-compile specialized functions. Julia does very little runtime magic to speed up code, especially compared to PyPy. The speed just comes from a very clever type system + LLVM.
- mafribe 11y agoThere is another use case for JIT compilers that has not been considered: speeding up programming language development. PyPy's meta-tracing JIT-compiler framework [1] allows you to generate reasonably fast tracing JIT compilers for languages essentially by writing an interpreter. Laurie Tratt has written a nice description of this, and the advantages this delivers [2]. This make the development of new programming languages much simpler, because you don't have to invest a lot of time and effort into producing a reasonably fast compiler (whether JIT or AOT) at the beginning of language development. Yes, in many cases you can build AOT compilers that beat JITs produced by meta-tracing an interpreter, but not easily so. Tratt's team has used this approach for building impressive multi-language tools [3] that allow you to write programs in heterogeneous languages like Python, Prolog and PHP at the same time. [1] http://stups.hhu.de/mediawiki/images/f/f5/Tracing_JITs11_tracing_the_meta_level.pdf http://stups.hhu.de/mediawiki/images/f/f5/Tracing_JITs11_tra... [2] http://tratt.net/laurie/blog/entries/fast_enough_vms_in_fast_enough_time http://tratt.net/laurie/blog/entries/fast_enough_vms_in_fast... [3] http://soft-dev.org/ http://soft-dev.org/
- fijal 11y ago"Yes, you can build AOT compilers that beat JITs produced by meta-tracing an interpreter" - not for all the languages. As mentioned below, there is a certain tradeoff associated with JITs - warmup time, memory consumptions etc. But for certain class of problems (say compiling Python) and for certain class of use cases (running at top speed), I dare you to compete with PyPy. The biggest contender so far comes from the Zippy project, which is indeed built on truffle which is a meta JIT, albeit a method based one.
- mwcampbell 11y agoI'm ambivalent about this subject. On the one hand, it intuitively makes sense that JIT compilation is inferior to AOT compilation for an application that will be packaged once on a developer's machine and then run on thousands or millions of devices. Doing the JIT compilation on all of those devices is a waste of energy; it's better to do the compilation just once on the developer's machine. Also, JIT compilation requires warm-up to get to good performance. The kind of consumer who won't pay more than a few dollars for an app will also be turned off by an app that isn't snappy right away. So the first impression is everything, and a warm-up period before good performance isn't as acceptable here as it is in server applications. On the other hand, on the major mobile platforms, native apps which can be AOT-compiled have to be distributed by gatekeepers, i.e. the app stores. On these platforms, the only alternative to the gatekeepers is the web. So I have a non-technical reason to want the last major JIT-compiled language on the client side, JavaScript, to be good enough for a variety of applications. I was going to comment on the advantages of JIT compilation with regard to polymorphism, but then I found that the OP addressed that with his argument about predictability being more important than best-case or even average-case performance. Finally, I'm curious if the author of the OP would use the same argument about predictability over best or average case performance to argue for reference counting over tracing GC. Maybe Android's ART and .NET Native would benefit from using reference counting in combination with a backup tracing GC to handle cycles, like CPython.
- pjmlp 11y ago.NET GC is quite good and they introduced features in version 4.6 that allow even for fine-grained control over when collections happen and how. Reference counting is only helpful if directly supported by the compiler, to remove inc/dec pairs. Also it is worse than GC for multi-threaded applications. Currently I think the only alternative to GC are Substructural type systems. Even C++ guys are now looking into this. The problem is making it palatable to the average mainstream programmer.
- wiz21c 11y agoIn my company, we develop on windows, build on linux and run on solaris (JavaEE). Moreover (forget devops!) the deployment team are not the development team. Having an environment (JVM + web server + our code) that is the same across the various hardware and company's teams is really something that helps. So besides the AOT/JIT discussion, the value of the JVM in itself is really good for us.
- pjmlp 11y agoYes, but that hasn't anything to do with AOT/JIT rather a portable binary format for Java executables. A few commercial JVM vendors do offer AOT compilation on their JDKs.
- vvanders 11y agoThe "hybrid" model they refer to is very common in Game-Dev. C/C++ down at the engine level tied together with Lua/UnrealScript/Lisp/etc.
- corysama 11y agoThis widespread move towards AOT is interesting in comparison to the prevailing attitude 5 years ago. "Have tracing jit compilers won?" http://lambda-the-ultimate.org/node/3851 http://lambda-the-ultimate.org/node/3851
- noblethrasher 11y agoAOTs are best for when you want to use universal computers to run computations (e.g. apps). JITs are best for when you want to use universal computers to run better universal computers.
- skybrian 11y agoIn the case of Android, the big downside is very slow system updates because all apps on the device have to be recompiled. But it probably can be fixed.