Maybe - it sounds tricky though, since optimizations can be rendered invalid across runs (e.g. a plugin may be loaded that invalidates a monomorphic dispatch op

Maybe - it sounds tricky though, since optimizations can be rendered invalid across runs (e.g. a plugin may be loaded that invalidates a monomorphic dispatch optimization). So you must track and re-validate the assumptions underlying the optimized code before it can be used. Also startup code is often executed once, and so is a poor candidate for expensive JIT optimizations. That said, in principle it seems like one could do as you say - it's just hard. Empirically, the industry seems to be moving from JIT to AOT, at least for clients. .NET -> Ngen, Dalvik -> ART, JavaScript -> asm.js, etc. (Oh, and it's a suspect assumption that there exists other cores that "would be idle anyways!")