That's exactly my point: you cannot abstract away details of pipeline size. You must know this to get the best performance out of your structures and algorithm

That's exactly my point: you cannot abstract away details of pipeline size. You must know this to get the best performance out of your structures and algorithms. There is no way for a compiler/optimizer to look at your container implementation and say "oh, this will potentially overflow the pipeline, let's split the data structure into multiple parts and change the code to handle this new access stategy"... that would be creating entirely new code (what would stepping through a debugger look like on such code?).