That's true, but most programs don't encounter situations where a whole new class of unforeseen optimizations are needed right now, /and/ it's possible to make

That's true, but most programs don't encounter situations where a whole new class of unforeseen optimizations are needed right now, /and/ it's possible to make a significant improvement without changing any algorithms. For everything else, there is Profile Guided Optimization.