I'm really interested in this space. I'm the original author of the Wikipedia page on SafeTSA. I really want compiling to native code to be as common as writi

I'm really interested in this space. I'm the original author of the Wikipedia page on SafeTSA. I really want compiling to native code to be as common as writing inline assembly currently is (and I see no fundamental reason why it won't eventually be). I used to be one of 5 developers for the most popular Java desktop program in the early 2000s. I have some ideas for hardware features to reduce the overhead of run-time code re-optimization based on real-time profiles. I guess when I eventually retire, I'll play around with an FPGA CPU implementation of my ideas. I really believe JITs can do what you're describing, but I think it's important not to over-sell current JIT implementations. Over-selling the technology results in skepticism. In my ideal world, we compile to either a high-level control-flow-graph/static-single-assignment representation (maybe similar to SafeTSA) as a distribution format. At installation time, we AoT-compile to native code (similar to the current Android runtime). At run-time, we take advantage of extremely low overhead hardware profiling/tracing features to re-optimize hot spots. Optimal ordering/inclusion of various optimizations is a bit of a black art, so I expect some degree of randomization here. In cases where profiling shows the re-optimized code out-performs the AoT version, we'd then cache the re-optimizations to disk. Optionally, a periodic process would periodically perform more expensive offline static-re-optimization of the AoT'd binary, similar to the current Android runtime. Crucially, the garbage collector should be decoupled from the binary generation/re-optimization system. The CPU overhead of garbage collection can be brought pretty low, but there's a size/space trade off. Last I checked, the rule of thumb is that unless you're manually pooling objects, you can expect a GC'd program to use roughly twice as much memory as an equivalent C++ program. Now, the cognitive/development overhead of manual memory management usually isn't worth the memory savings. However, I've worked in several domains where the memory savings are worth the slower development time.