Sure, in lucky situations I've been able to speed things up by an order of magnitude by just changing memory layout and using SSE. But 46% in a tight spot is wo

Sure, in lucky situations I've been able to speed things up by an order of magnitude by just changing memory layout and using SSE. But 46% in a tight spot is worthwhile change. Basically, when dealing with a highly performance intensive code (storage, if you must know), my rule of thumb was, 2% of more of improvement in overall end-to-end throughput on a long running benchmark is worthwhile if change is not too complicated. This could mean 10x improvement in one of the parts of the pipeline, or 2% improvement across the board, or anything in between: the goal was overall throughput. Likewise, nothing that regressed the performance could ever go in. You'd be surprised what you can squeeze out of your code if you establish these ground rules and run with them for a year.