Your point that the application level programmer should not be burdened with details like CPU cache size is something the article's author may agree with. They

Your point that the application level programmer should not be burdened with details like CPU cache size is something the article's author may agree with. They mentioned that cramming 17 floats into a cache that can hold 16 is (premature) optimization. On the other hand, the author is arguing that developers should take the presence of a CPU cache into consideration. The distinction is important. By failing to take the hardware into consideration it becomes much more difficult to solve performance issues down the road since an avenue to resolve it will likely involve rewriting disparate parts of code. Incidentally, this type of issue is unlikely to be addressed by optimizing compilers since it requires a knowledge of the runtime performance of disparate parts of code and may involve code that is not compiled at build time (e.g. the data structure may be passed to a third-party library).