4 ms·

> Secondly, we have got machines equipped with multi-level stores, presenting us problems of management strategy that, in spite of the extensive literature on t

> Secondly, we have got machines equipped with multi-level stores, presenting us problems of management strategy that, in spite of the extensive literature on the subject, still remain rather elusive. NUMA only got more complicated over time. The range of latency differences is more extreme than ever. We've got L1 running at nanosecond delay, and on the other end we've got cold tapes that can take a whole day to load. Which kind of memory/compute to use in a heterogeneous system (cpu/gpu) is also something that can be difficult to figure out. Multi core is likely the most devastating dragon to arrive since this article was written. Premature optimization might be evil, but it's the only way to efficiently align the software with the memory architecture. E.g., in a Unity application, rewriting from game objects to ECS is basically like starting over. If you could only focus on one aspect, I would keep the average temperature of L1 in mind constantly. If you can keep it semi-warm, nothing else really matters. There are very few problems that a modern CPU can't chew through ~instantly assuming the working set is in L1 and there is no contention with other threads. This is the same thinking that drives some of us to use SQLite over hosted SQL providers. Thinking in terms of not just information, but the latency domain of the information, is what can unlock those bananas 1000x+ speed ups.

kragen 1y ago
> If you could only focus on one aspect, I would keep the average temperature of L1 in mind constantly. If you can keep it semi-warm, nothing else really matters. This is easy, as Dijkstra points out. Allocate an array ¾ the size of L1D, and write a subroutine to reverse its contents, but don't use the array otherwise. Call the useless reversal subroutine often enough (at the beginning of every other subroutine and loop body should generally be enough) that the CPU spends most of its time running it. That will ensure that most of your data references are to the array, and most of your instruction references are to the reversal subroutine, both of which are always in cache. Other things do matter.
pjmlp 1y ago
This is why having language runtimes that are NUMA aware like in JVM implementations (-XX:+UseNUMA) or languages like Chapel, is much easier than requiring everyone to be a top coder. It isn't the same as manually doing the optimizations, still half way there is better than not at all.

kragen 1y ago

> If you could only focus on one aspect, I would keep the average temperature of L1 in mind constantly. If you can keep it semi-warm, nothing else really matters. This is easy, as Dijkstra points out. Allocate an array ¾ the size of L1D, and write a subroutine to reverse its contents, but don't use the array otherwise. Call the useless reversal subroutine often enough (at the beginning of every other subroutine and loop body should generally be enough) that the CPU spends most of its time running it. That will ensure that most of your data references are to the array, and most of your instruction references are to the reversal subroutine, both of which are always in cache. Other things do matter.

pjmlp 1y ago

This is why having language runtimes that are NUMA aware like in JVM implementations (-XX:+UseNUMA) or languages like Chapel, is much easier than requiring everyone to be a top coder. It isn't the same as manually doing the optimizations, still half way there is better than not at all.