3 ms·

Having glanced at the code, it seems fairly easy to extend to modern Intel CPUs, which is great. The biggest caveat is the lack of prefetcher simulation, which

Having glanced at the code, it seems fairly easy to extend to modern Intel CPUs, which is great. The biggest caveat is the lack of prefetcher simulation, which literally makes many seemingly horrible cache misses a non-issue, but that's a non-trivial extension. Regardless, very nice to see tools like this being open sourced. The games industry has a lot of valuable experience with performance-critical code and I'd love to see more of it shared publicly.

gchadwick 10y ago
> The biggest caveat is the lack of prefetcher simulation Certainly an important point, but only relevant if what you are doing had a hope of being prefetched in the first place. Maybe the tool could be extended to give an idea of how 'prefetchable' something is? This should be simpler than a full prefetcher simulation. Then you can concentrate on the cache misses it predicts that are unlikely to prefetched. The tool is more focussed on pointing out areas you can improve, it doesn't matter if it gets it wrong sometimes.
daemin 10y ago
If you use this tool to find the outliers which just break the cache badly, then you can apply some human introspection on the data and figure out what is a problem and what is not. Trying to add in a prefetch handler would be painful, time consuming, and may actually hide some cache misses that you would want to find. Also each CPU family and potentially in between families there could be changes to the prefetch algorithm used.
- keldaris 10y ago
  Very true, hence "caveat" not "problem" or "dealbreaker". It's something you have to constantly keep in mind as you look at the results, some very fast array iterations will habitually look terrible, etc. Given the features of modern prefetchers, much less obvious patterns will also look worse than they are every now and then.
Twirrim 10y ago
Is it bad to try to avoid relying on the pre-fetcher? It's obviously there for a good reason and presumably could cut down on a bunch of micro-optimisation, I'm just curious how much of a significant performance measuring black hole it's leaving by not considering it.
- keldaris 10y ago
  Depending on your workload deliberately avoiding the prefetcher (which isn't particularly easy, mind you) ranges between a mild loss to 2-3 orders of magnitude in performance. As a trivial example, consider the element-wise addition of two large matrices. Without a prefetcher you would incur a large number (how large mostly depends on the ratio of the number of elements in your matrices and the available cache memory) of cache misses even during the most cache-friendly iteration pattern since the matrices are simply too large. On modern CPUs the prefetcher avoids these cache misses so well that the performance difference is approximately two orders of magnitude. Of course for this to work your traversal pattern has to be sufficiently predictable. Generally, I think considering the prefetcher is roughly as important (and, indeed, inextricably linked) as considering the cache subsystem itself. Any performance sensitive code will benefit from careful consideration of memory access and traversal patterns.

gchadwick 10y ago

> The biggest caveat is the lack of prefetcher simulation Certainly an important point, but only relevant if what you are doing had a hope of being prefetched in the first place. Maybe the tool could be extended to give an idea of how 'prefetchable' something is? This should be simpler than a full prefetcher simulation. Then you can concentrate on the cache misses it predicts that are unlikely to prefetched. The tool is more focussed on pointing out areas you can improve, it doesn't matter if it gets it wrong sometimes.

daemin 10y ago

If you use this tool to find the outliers which just break the cache badly, then you can apply some human introspection on the data and figure out what is a problem and what is not. Trying to add in a prefetch handler would be painful, time consuming, and may actually hide some cache misses that you would want to find. Also each CPU family and potentially in between families there could be changes to the prefetch algorithm used.

keldaris 10y ago
Very true, hence "caveat" not "problem" or "dealbreaker". It's something you have to constantly keep in mind as you look at the results, some very fast array iterations will habitually look terrible, etc. Given the features of modern prefetchers, much less obvious patterns will also look worse than they are every now and then.

Twirrim 10y ago

Is it bad to try to avoid relying on the pre-fetcher? It's obviously there for a good reason and presumably could cut down on a bunch of micro-optimisation, I'm just curious how much of a significant performance measuring black hole it's leaving by not considering it.

keldaris 10y ago
Depending on your workload deliberately avoiding the prefetcher (which isn't particularly easy, mind you) ranges between a mild loss to 2-3 orders of magnitude in performance. As a trivial example, consider the element-wise addition of two large matrices. Without a prefetcher you would incur a large number (how large mostly depends on the ratio of the number of elements in your matrices and the available cache memory) of cache misses even during the most cache-friendly iteration pattern since the matrices are simply too large. On modern CPUs the prefetcher avoids these cache misses so well that the performance difference is approximately two orders of magnitude. Of course for this to work your traversal pattern has to be sufficiently predictable. Generally, I think considering the prefetcher is roughly as important (and, indeed, inextricably linked) as considering the cache subsystem itself. Any performance sensitive code will benefit from careful consideration of memory access and traversal patterns.