13 ms·
I remember years ago when Java adjacent research was all the rage, HP had a problem that was “Rosetta lite” if you will. They had a need to run old binaries on
by hinkley 4y ago
I remember years ago when Java adjacent research was all the rage, HP had a problem that was “Rosetta lite” if you will. They had a need to run old binaries on new hardware that wasn’t exactly backward compatible. They made a transpiler that worked on binaries. It might have even been a JIT but that part of the memory is fuzzy.
What made it interesting here was that as a sanity check they made an A->A mode where they took in one architecture and spit out machine code for the same architecture. The output was faster than the input. Meaning that even native code has some room for improvement with JIT technology.
I have been wishing for years that we were in a better place with regard to compilers and NP complete problems where the compilers had a fast mode for code-build-test cycles and a very slow incremental mode for official builds. I recall someone telling me the only thing they liked about the Rational IDE (C and C++?) was that it cached precompiled headers, one of the Amdahl’s Law areas for compilers. If you changed a header, you paid the recompilation cost and everyone else got a copy. I love whenever the person that cares about something gets to pay the consequence instead of externalizing it on others.
And having some CI machines or CPUs that just sit around chewing on Hard Problems all day for that last 10% seems to be to be a really good use case in a world that’s seeing 16 core consumer hardware. Also caching hints from previous runs is a good thing.
- fuckstick 4y ago> The output was faster than the input. So if you ran the input back through the output multiple times then that means you could eventually get the runtime down to 0.
- deleted 4y ago[deleted]
- avidiax 4y agoProbably the output of the decade-old compiler that produced the original binary had no optimizations.
- hinkley 4y agoThat too but the eternal riddle of optimizer passes is which ones reveal structure and which obscure it. Do I loop unroll or strength reduce first? If there are heuristics about max complexity for unrolling or inlining then it might be “both”. And then there’s processor family versus this exact model.
- twic 4y agoBut unfortunately, the memory use goes to infinity.
- zaphirplane 4y agoIs this for itanium
- vinyl7 4y ago> where the compilers had a fast mode for code-build-test cycles and a very slow incremental mode for official builds. That already exists with c/c++...unity builds aka single translation unit builds. Compiling and linking a ton of object files takes an inordinate amount of time, often the majority of the build time
- bogomipz 4y ago>"I remember years ago when Java adjacent research was all the rage, ..." What is meant by "Java adjacent research"? I'm not familiar with what that was.
- wmf 4y agohttps://www.hpl.hp.com/techreports/1999/HPL-1999-78.html https://www.hpl.hp.com/techreports/1999/HPL-1999-78.html
- travisgriggs 4y agoIt was particularly poignant at the time because JITed languages were looked down on by the “static compilation makes us faster” crowd. So it was a sort of “wait a minute Watson!” moment in that particular tech debate. No one cares as much now days, we’ve moved our overrated opinion battlegrounds to other portions of what we do.
- kllrnohj 4y agoI think you're over estimating the impact or relevance of that anecdote. Particularly since the "static compilation makes us faster" crowd turned out to be correct, and people use JITs for non-performance reasons and just pay the performance tax they so often come with. The time-constrained nature in which a JIT has to run just largely kills it's theoretical runtime information gathering advantages. Devirtualization remains about the most advanced trick in the JIT book, which is generally not an issue static compiled languages struggle with in the first place.
- pjmlp 4y agoSure, because everyone ships static linked binaries.
- pjmlp 4y agoI eventually changed my opinion into JIT being the only way to make dynamic languages faster, while strong typed ones can benefit from having both AOT/JIT for different kinds of deployment scenarios, and development workflows.
- mikepurvis 4y agoI think I landed in a place where it's basically "the compiler has insufficient information to achieve ideal optimization because some things can only be known at runtime." Which is not exclusively an argument for runtime JIT— it can also be an argument for instrumenting your runtime environment, and feeding that profiling data back to the compiler to help it make smarter decisions the next time. But that's definitely a more involved process than just baking it into the same JavaScript interpreter used by everyone— likely well worth it in the case of things like game engines, though.
- jeffbee 4y agoPost-build optimization of binaries without changing the target CPU is common. See BOLT https://github.com/facebookincubator/BOLT https://github.com/facebookincubator/BOLT
- freedomben 4y agoIf JIT-ing a statically compiled input makes it faster, does that mean that JIT-ing itself is superior or does it mean that the static compiler isn't outputting optimal code? (real question. asked another way, does JIT have optimizations it can make that a static compiler can't?)
- vips7L 4y agoYes, the JIT has more profile guided data as to what your program actually does at runtime, therefore it can optimize better.
- gpderetta 4y agoOn the other hand some optimization are so expensive that a JIT just doesn't have the execution budget to perform them. Probably the optimal system is an hybrid iterative JIT/AOT compiler (which incidentally was the original objective of LLVM).
- andrewaylett 4y agoIt depends on what the JIT does exactly, but in general yes a JIT may be able to make optimisations that a static compiler won't be aware of because a JIT can optimise for the specific data being processed. That said, a sufficiently advanced CPU could also make those optimisations on "static" code. That was one of the things Transmeta had been aiming towards, I think.
- kmeisthax 4y agoIt's more the case that the ahead-of-time compilation is suboptimal. Modern compilers have a thing called PGO (Profile Guided Optimization) that lets you take a compiled application, run it and generate an execution profile for it, and then compile the application again using information from the profiling step. The reason why this works is that lots of optimization involves time-space tradeoffs that only make sense to do if the code is frequently called. JIT only runs on frequently-called code, so it has the advantage of runtime profiling information, while ahead-of-time (AOT) compilers have to make educated guesses about what loops are the most hot. PGO closes that gap. Theoretically, a JIT could produce binary code hyper-tailored to a particular user's habits and their computer's specific hardware. However, I'm not sure if that has that much of a benefit versus PGO AOT.
- hamstergene 4y agoCould it be simply because many binaries were produced by much older, outdated optimizers. Or optimized for size. Also, optimizers usually target “most common denominator” so native binaries rarely use full power of current instruction set. Jumping from that peculiar finding to praising runtime JIT feels like a longshot. To me it’s more of an argument towards distributing software in intermediate form (like Apple Bitcode) and compiling on install, tailoring for the current processor.
- kllrnohj 4y ago> To me it’s more of an argument towards distributing software in intermediate form (like Apple Bitcode) and compiling on install, tailoring for the current processor. Or distribute it in source form and make compilation part of the install process. Aka, the Gentoo model.
- jasonwatkinspdx 4y agoAll reasonable points, but examples where JIT has an advantage are well supported in research literature. The typical workload that shows this is something with a very large space of conditionals, but where at runtime there's a lot of locality, eg matching and classification engines.
- astrange 4y ago> To me it’s more of an argument towards distributing software in intermediate form (like Apple Bitcode) and compiling on install, tailoring for the current processor. This turns out to be quite difficult, especially if you're using bitcode as a compiler IL. You have to know what the right "intermediate" level is; if assumptions change too much under you then it's still too specific. And it means you can't use things like inline assembly. That's why bitcode is dead now. By the way, I don't know why this thread is about how JITs can optimize programs when this article is about how Rosetta is not a JIT and intentionally chose a design that can't optimize programs.
- josephg 4y ago> That's why bitcode is dead now. WebAssembly seems alive and well. I'm not sure how similar it is to java bitcode but its the same core idea. That said, WASM through v8 is ~3x slower than the same code compiled natively. (Some of this might be due to the lack of SIMD in wasm).
- mark_undoio 4y agoSomething that fascinates me about this kind of A -> A translation (which I associate with the original HP Dynamo project on HPPA CPUs) is that it was able to effectively yield the performance effect of one or two increased levels of -O optimization flag. Right now it's fairly common in software development to have a debug build and a release build with potentially different optimisation levels. So that's two builds to manage - if we could build with lower optimisation and still effectively run at higher levels then that's a whole load of build/test simplification. Moreover, debugging optimised binaries is fiddly due to information that's discarded. Having the original, unoptimised, version available at all times would give back the fidelity when required (e.g. debugging problems in the field). Java effectively lives in this world already as it can use high optimisation and then fall back to interpreted mode when debugging is needed. I wish we could have this for C/C++ and other native languages.
- saagarjha 4y agoIt depends greatly on which optimization levels you’re going through. —O0 to -O1 can easily be a 2-3x performance improvement, which is going to be hard to get otherwise. -O2 to -O3 might be 15% if you’re lucky, in which case -O+LTO+PGO can absolutely get you wins that beat that.
- bluGill 4y ago-O2 to -O3 has in some benchmarks made things worse. In others it is a massive win, but in generally going above -O2 should not be done without bench marking code. There are some optimizations that can make things worse or better for reasons that compiler cannot know.
- astrange 4y agoOver-optimizing your "cold" code can also make things worse for the "hot" code, eg by growing code size so much that briefly entering the cold space kicks everything out of caches.
- 4y ago
- tomcam 4y agoI’m likely misunderstanding what you said, but I thought pre-compiled headers were pretty much standard these days.
- tomcam 4y agoWhat on earth did I say to merit the downvotes?
- chrisseaton 4y agoI've run Ruby C extensions on a JIT faster than on native, due to things like inlining and profiling working more effectively at runtime.
- sergimas15 4y ago
- hawflakes 4y agoPeople have mentioned the Dynamo project from HP. But I think you're actually thinking of the Aries project (I worked in a directly adjacent project) that allowed you to run PA-RISC binaries on IA-64. https://nixdoc.net/man-pages/HP-UX/man5/Aries.5.html https://nixdoc.net/man-pages/HP-UX/man5/Aries.5.html
- AussieWog93 4y agoOutside of gaming, or hyper-CPU-critical workflows like video editing, I'm not really sure if people actually even care about that last 10% of performance. I know most of the time I get frustrated by everyday software, its doing something unnecessary in a long loop, and possibly forgetting to check for Windows messages too.
- koala_man 4y agoPerformance also translates into better battery life and cheaper datacenters.