5 ms·
This is one of the reasons why I think microbenchmarks are either not very useful or completely misleading --- they'll favour things like insanely unrolled loop
by userbinator 6y ago
This is one of the reasons why I think microbenchmarks are either not very useful or completely misleading --- they'll favour things like insanely unrolled loops or aligned data with lots of padding, when in fact on a macro-scale those things will certainly affect performance negatively due to cache misses.
My preferred approach when optimising is to go for size first; then if more speed is desired, carefully apply size-increasing optimisations to areas which may benefit, testing with a macro benchmark to judge any improvement.
- jhallenworld 6y agoI've found an interesting converse with macro benchmarks: small improvements to any of the top functions shown in your profiler output will have a relatively larger than expected effect on the overall speed. I mean you make a change that improves just one function by a little bit on its own (within the benchmark), but the overall is better than this. I think you end up freeing resources used by everything else. The trick is to optimize the right macro benchmark- one that matches your customer's key use case I suppose.
- YZF 6y agoIf the macro scale is dominated by the code you're optimizing, which is often the case, let's say some inner loops of a codec, signal processing, etc. then the cache misses that may follow aren't really a concern. Obviously(?) taking some random code that doesn't run frequently and trying to optimize it won't make a difference. Microbenchmarks are pretty useful in this context since they let you iterate faster. More generally the question is what is the bottleneck, if the bottleneck is fetching stuff into caches then that's what you optimize for, if the bottleneck is some computation that is running from caches then that's what you optimize...
- jeffbee 6y agoThe big cautionary tale of microbenchmarking is memcpy. If you look at the Gnu memcpy, it's ridiculously long and obviously been guided by extensive benchmarking. And memcpy often shows up in profiles, so it is a big target. But in real systems a small simple loop will perform better due to icache pressure.
- api 6y agoIt's amazing to me that every CPU doesn't have specialized memory copying instructions given how common an operation this is and how it can be a bottleneck. We have hundreds of vector instructions to optimize all kinds of special cases but little for this most general thing?
- userbinator 6y agox86 does, it's called REP MOVS, and it copies entire cachelines at a time (if the size of the copy lets it.) AFAIK it has done this since the P6 family (Pentium II), and has only gotten better over time. As a bonus, it is only 2 bytes long. I believe the Linux kernel uses it as its default memcpy()/memmove(). Here's some very interesting discussion about it: https://stackoverflow.com/questions/43343231/enhanced-rep-movsb-for-memcpy/43574756#43574756 https://stackoverflow.com/questions/43343231/enhanced-rep-mo...
- api 6y agoWhy isn't it the default everywhere?
- jeffbee 6y agoAs I tried to indicate, the microbenchmark-guided unrolled loops look better than the simple `rep` instruction because the benchmarks aren't realistic. They don't suffer from mispredicted branches, they don't experience icache pressure. Basically the microarchitectural reality is absent in a small benchmark. There's also the fact that `rep` startup cost was higher in the past than it is now. I think it started to get really fast around Ivy Bridge. This is all discussed in Intel's optimization manual, by the way.
- xelxebar 6y agoNeat. I was unaware of this manual. For the lazy but curious: https://software.intel.com/content/www/us/en/develop/download/intel-64-and-ia-32-architectures-optimization-reference-manual.html https://software.intel.com/content/www/us/en/develop/downloa... And if you want ALL THE THINGS: https://software.intel.com/content/www/us/en/develop/articles/intel-sdm.html https://software.intel.com/content/www/us/en/develop/article...
- deleted 6y ago[deleted]
- GuB-42 6y ago> My preferred approach when optimising is to go for size first Agreed, in my experience, more often than not, size is speed. Small memory usage mean less cache misses, smaller memcpys, more chance of having data fit a single word, etc...
- scoutt 6y agoExcept when loops unrolls make things faster.
- ants_a 6y agoThat only tends to be the case for tiny loops where loop iterator updates can be folded into addressing displacements and that frees up an execution port needed for something else. If you don't know what you are doing unrolling is just as likely to hurt performance because you don't fit into uOP cache and get less decode bandwidth as a result. Or you increase ICache pressure on macro benchmarks and hurt real world performance. Modern cores are really good at hiding loop accounting overhead.
- scoutt 6y agoIn a world where all the electronic devices are composed of x86 cores and/or have a data and/or instruction cache, it could be true. But there are plenty of architectures out there where space-time tradeoffs (by optimized compilers) are still a thing (my PoV is from the embedded industry).