From the late 90s until 2006, I believed in the sufficiently smart JIT. There were a few benchmarks showing corner cases of Java out-performing C/C++. I follo

From the late 90s until 2006, I believed in the sufficiently smart JIT. There were a few benchmarks showing corner cases of Java out-performing C/C++. I followed the research. I was particularly excited by Michael Franz's SafeTSA research on more JIT-optimized program representations. Surely we were just a few years from Java generally out-performing C/C++! I worked on the most popular Java desktop app. Then I moved to Google, working on the indexing system and properly learned C++. I've since worked on several projects for several companies where the annual cost for compute time is easily 200 times my salary. These are systems where it pays to extensively profile both CPU and I/O and retain C++ optimization experts. I still see the appeal of platform-independent binaries and dynamic optimization, but you really need the startup time and optimization cost advantages of ahead-of-time compilation. Hopefully the platform independence and dynamic optimization features would also be decoupled from the garbage collector. HP Research had project dynamo that was basically a tracing JIT emulator for PARisc CPUs that ran on PARisc. It showed that binaries compiled with -O2 could get performace comparable to -O4 through dynamic recompilation. Ideally, we'd distribute programs in a compact and compressed CPU-independent SSA representation similar to SafeTSA or LLVM bitcode. Installation would AoT-compile the binaries and keep around the SSA form for use by an HP Dynamo-like runtime optimizer. The AoT could compile functions and methods to lists strait-line extended basic blocks with a call to a trampoline loop or other techniques for lightweight instrumentation/tracing of native code. The dynamic optimizer wouldn't need to work with arbitrary machine code, only AoT compiler output. Also, it would never have to disassemble native code, but could always start by gluing together the SSA for each extended basic block in the trace. A few CPU features could make native tracing and dynamic recompilation extremely light weight. Unfortunately, Intel and ARM are disincentivised to do so, as it would make it easier to migrate off of their intellectual property.