4 ms·
16 years, I created a profiling macro system on for select set critical functions. The profiling system can switch the measuring data from #instruction, #cpu_
by tkinom 11y ago
16 years, I created a profiling macro system on for select set critical functions. The profiling system can switch the measuring data from #instruction, #cpu_clk, #branch_stall_cycles, #L1_miss, #L2 miss for all the critical path functions.
After analyzed the data, I found branch stall cycles and data access stall cycles were causing huge number of delays in the critical code path.
I used the following tricks to get rid of the stall cycles.
1) Use branch_likely to force gcc to make sure there is no branch at all in the critical path of executions. (save 30+ cpu cycle per branch, there are a lot of branch stall cycles if one just simplely follow the gcc generated "optimized" code. MIPS CPU 200Mhz)
2) Use prefetch ahead of data structure access to get rid of un-cache data delay. (save ~50+cpu cycle per data stall, also, there are lot of them in the critical path.)
3) Use inline functions, etc to get rid of call stalls in critical path.
The system got ~100x increase on the overall system thru-put with those techniques with just pure C optimization from standard -O2 build.
I think it might be possible to create a build system that can automatically collect the profiling data (branch stall cycles and data stall cycles) and use the branch likely and prefetch instructions to auto-optimized the critical path code.
Specifying which code path / function call sequences are the real critical path probably still require programmer's touch.
As result of using data prefetch code in proper place, I don't used cache locking nor doing any kind of CPU affinity trick to generated the optimized obj code without any stall cycles for critical code path.
- pmalynin 11y agoSo...profiler guided optimization lol.
- devbug 11y agoYeah. Nothing ground-breaking here. Would still love to have a great open-source tool for this.
- cbsmith 11y agoYou mean like oprofile, dtrace, system tap, cachegrind, etc.?
- devbug 11y agoAh, I see where you may have got confused. I meant regarding the automatic application of rules based on profiling.
- wtallis 11y agoWhat are you looking for that is not currently part of the profile guided optimization capabilities of current compilers? It has been the case for years that you can compile a program with profiling/tracing hooks, run it through a representative workload, and feed the profiler output back to the compiler to get a faster executable. That's why no Firefox you compile yourself will outperform the one you download from Mozilla, if you only compile it a single time.
- uxcn 11y ago> I think it might be possible to create a build system that can automatically > collect the profiling data (branch stall cycles and data stall cycles) > and use the branch likely and prefetch instructions to auto-optimized the > critical path code. Recent versions of Clang and GCC actually support profile guided optimizations using gcov format files to trace executions. There are a number optimizations that the compiler can determine profitable knowing how something typically executes. They don't use sampled or simulated metrics though, and I'm not sure if they add prefetch or non-temporal optimizations. The ones you mentioned aren't exactly equivalent to cache partitioning (etc...) though. Partitioning allows explicit allocation and prioritization of a contended resource instead of only improving utilization for a single process. So, for example if two threads/processes have an 8MiB working set, and they run on the same cpu, they can easily step on eachother's data in the cache. If you partition it though, less frequently used data doesn't have to step on more frequently used data.
- xavierd 11y agoA lot of those optimizations would no longer yield any benefits[0]. The CPU archictecture evolved a lot in 16 years, especially in branch/code prediction to the point where a correctly predicted branch (without branch_likely) has almost no cost. [0]: At least, this is true for x86 CPUs.
- userbinator 11y agoDitto for prefetch instructions: https://lwn.net/Articles/444336/ https://lwn.net/Articles/444336/ A MIPS is probably the exact opposite to modern (which actually means anything P6 and above) x86 CPUs in terms of performance characteristics. If I were to guess what member of the x86 family might actually benefit from such optimisation, it would be NetBurst (which itself has very different performance characteristics from every other x86 family that came before or after it.)
- tkinom 11y agoI was trying to optimize for a network app. The goal of trying to get to 1 million pps. At that time 200Mz CPU, 1 cache miss is 50+ cycles. or 25% of the CPU budget, prefetch helped a lot in that case.
- kbenson 11y agoI've occasionally wondered how long it takes highly optimized C/C++ to be surpassed by optimizing compilers due to CPU advancement and the optimizations either making compiler optimization harder, or the optimizations target assumptions about CPU architecture that are no longer valid. That is, what is the shelf life of a very low level CPU optimization for Intel hardware.
- daemin 11y agoWell while strictly not on topic, there was this talk recently on micro-optimisations. https://www.youtube.com/watch?v=nXaxk27zwlk https://www.youtube.com/watch?v=nXaxk27zwlk