It can be tough even without the impact of the memory hierarchy. I've seen code where adding an extra instruction to the calculation made it faster. The extra i

It can be tough even without the impact of the memory hierarchy. I've seen code where adding an extra instruction to the calculation made it faster. The extra instruction implicitly eliminated denormals, thus resulting in faster execution with some workloads on systems where operations on denormal values were slower than operations on other values. It was a completely unnecessary instruction from a correctness perspective, because it had no effect on the answer. However, it was important for performance; removing the instruction made the calculation slower.