6 ms·
Stupid question for people who know more about these things. Why can’t we fight the warmup time by running an already warmed up snapshot of the program? Or say
by Spivak 7y ago
Stupid question for people who know more about these things. Why can’t we fight the warmup time by running an already warmed up snapshot of the program? Or say dumping some data structure when it hits steady state to give hints to the JIT the next time it runs?
- pizlonator 7y agoThat would be great but it’s hard because the trick, at least in JS VMs, is to have the JIT specialize based on the heap. So you’d need a heap snapshot or some way to link the generated code to a different heap. Maybe not impossible, just hard enough that it’s not widespread.
- hamburglar 7y agoIt does seem like you could do JIT with a persistent cache which stores the JIT output along with a key that's a hash of all the relevant system parameters like CPU model and VM parameters like heap size. This would mean that the typical case of re-running a program in the same environment would be pre-warmed.
- rovr138 7y agoKind of like memchached, https://en.wikipedia.org/wiki/Memcached https://en.wikipedia.org/wiki/Memcached
- pizlonator 7y agoIt’s much harder than that because the JIT is speculating on what lots of objects in the heap are doing, including watchpointing them to constant fold properties. It’s not clear what the key should be in that case. Still not impossible but I want to be clear on what exactly makes this hard. CPU model for example is not what makes it hard.
- saagarjha 7y ago> including watchpointing them I assume this isn't literally using hardware watchpoints?
- pizlonator 7y agoYeah
- fulafel 7y agoHeap layout changes between runs due to OS obfuscation (ASLR).
- pizlonator 7y agoI would bet that a VM isn’t going to be deterministic enough to produce the same heap twice even without aslr. Just building a deterministic JS engine seems like a super hard problem.
- blinkingled 7y agoIBM OpenJ9 JVM does that with the AOT feature. It's good for startup time and the AOT code can be still further optimized by the JIT. https://www.eclipse.org/openj9/docs/aot/ https://www.eclipse.org/openj9/docs/aot/
- anp 7y agoIIRC V8 can now provide deterministic JIT+heap images in addition to preparsed bytecode images. EDIT: worth noting that other JS VMs like JSC have these APIs for bytecode too, but i can’t remember off hand whether they can do generated code too.
- deleted 7y ago[deleted]
- cortesoft 7y agoIsn't that just compiling, then?
- dodobirdlord 7y agoSure, but something like Java already has to be compiled. Throw a few more minutes in there, maybe run the test suite a couple thousand times.
- thayne 7y agoBut many languages with VMs are intended to be portable across processor architectures, and a snapshot would be architecture-dependent. Also, the charecteristics of your test suite may be very different than how it is run in production.
- afiori 7y agoI would say those are reasonably easy to solve, you just need to offer platform specific trained binaries and add a realistic set of stress tests with which you can train the VM.
- jakear 7y agoI believe this is what Deno tries to do. https://deno.land/ https://deno.land/
- electrum 7y agoAzul's ReadyNow! for their commercial Zing JVM does this: https://www.azul.com/products/zing/readynow-technology-for-zing/ https://www.azul.com/products/zing/readynow-technology-for-z...
- lstamour 7y agoIf you look for "AOT" or "Ahead-of-time" you'll find examples in both .NET and Java, but as far as I know they're either largely experimental or limited to newer code (not backwards-compatible with all code). But I haven't looked too deeply into it. Dumping data structures to give hints for next time reminds me of something I read recently on the topic, but drat, I can't remember it right now.
- jayd16 7y agoAndroid does this with ART. They use JIT and profiling to generate AOT binaries dynamically. I believe this also lets Android update the runtime in a way that invalidates the AOT binaries. They are simply be regenerated as needed. https://source.android.com/devices/tech/dalvik/jit-compiler https://source.android.com/devices/tech/dalvik/jit-compiler
- fulafel 7y agoWe can and there are many implementations of it, but the relatively rare application of this approach suggests it's not a big enough win to justify the cost.
- marcoperaza 7y agoEmacs does this. See https://emacs.stackexchange.com/questions/2364/what-is-the-function-purecopy-needed-for https://emacs.stackexchange.com/questions/2364/what-is-the-f...
- rb808 7y agodotnet does this, caches compiled assemblies then also tunes them. https://www.geeksforgeeks.org/what-is-just-in-time-jit-compiler-in-dot-net/ https://www.geeksforgeeks.org/what-is-just-in-time-jit-compi...
- krona 7y agoThe V8 snapshot feature is designed for this. https://v8.dev/blog/custom-startup-snapshots https://v8.dev/blog/custom-startup-snapshots
- The_rationalist 7y agohttps://github.com/facebook/nailgun https://github.com/facebook/nailgun