You can "go faster" than memory bandwidth by accessing memory more intelligently (that is, by reducing the amount of memory traffic, possibly by improving cache

You can "go faster" than memory bandwidth by accessing memory more intelligently (that is, by reducing the amount of memory traffic, possibly by improving cache behaviour). The classic example is loop tiling for matrix multiplication. Here you write significantly more code, but reduce accesses to main memory by a good constant factor. Sadly, with modern architectures, just keeping things small and simple is rarely the way to peak performance. My experience from doing high-performance programming (mostly on GPUs) is that you have a very generous budget when it comes to how large your code can be, but you have an extremely tight budget when it comes to how large your data is. (Of course, in absolute terms you access significantly more data than code, but you generally mostly worry about data size, not so much code size.)