I also had in mind branch prediction units, and stuff like Intel's TSX. I assume these days the former bleeds into many other parts of the architecture, includi

I also had in mind branch prediction units, and stuff like Intel's TSX. I assume these days the former bleeds into many other parts of the architecture, including the microcode; I'm certain (but can't say authoritatively) that the latter does. From a wholistic viewpoint I think it's fair to characterize modern CPUs as JIT-like. It's easy to trivialize individual aspects of the whole. You can do the same for software JITs. They're not magic (notwithstanding the breathless claims and river of thesis papers), and often simpler is better, anyhow (e.g. LuaJIT). For obvious reasons the tactics employed by CPUs will tend toward the simpler end of the spectrum.