5 ms·
Avoiding Instruction Cache Misses
- AnanasAttack 7y agoI've been wondering about the impact of insturction cache misses and potential downsides of executable bloat caused by C++ templates, but have never seen a real case of this being a problem.
- tempguy9999 7y agoHow would you know it's happening? How would you know it's not? How have you measured/looked? I'm know little about this area, so I'm curious.
- opencl 7y agoSeveral popular profiler tools (valgrind, visual studio profiler, dtrace, perfmon) can track cache miss rates.
- namibj 7y agoperf-tools are my favorite. The overhead is negligible, and thus any metrics you gather are very accurate. Valgrind is rarely useful, considering the execution time disadvantage.
- namibj 7y agoperf record -e icache.ifdata_stall ./test-program-binary followed by perf report or, for fancy viewing, use hotspot from KDAB. This is on linux, btw, but it should work fine with wine.
- jandrewrogers 7y agoModern C++ compilers are pretty good at minimizing template bloat. As with most things in C++ it requires some awareness and intent to optimize this but it isn't the issue it used to be in many cases.
- jcelerier 7y agoI don't understand where does this sentiment come from. I've once seen a complete high-level networking algorithm implemented with boost.asio be reduced to less than 250 instructions...
- mhh__ 7y agoExecutable bloat can be avoided by stripping the executable afterwards, if that's what you mean: Although the function body will have been provided if instantiated and used, the compiler will have probably inlined a huge amount of any code that is actually used - a lot of templated stuff is made up of one liners or single uses, which compilers are very very good about reasoning through the motion of these days. I guess a naive compiler could have issues but every time I check something on compiler explorer, the modern big boys (GCC, LLVM, ICC?) are pretty shrewd (Especially if you optimise for size).
- cbetti 7y agoWhile aggressive inlining eliminates function call overhead, it can actually exacerbate instruction cache misses because it makes the executable larger.
- cbetti 7y agoIn 2010 Endeca's analytics engine exe was growing substantially via introduction of template instantiations. The engine eventually crossed the threshold of instruction cache misses outweighing the benefits of avoiding type lookups at runtime, and the team agreed to stop pursuing template instantiation so aggressively as a performance enhancement. I wonder if modern compilers would have allowed the team to keep at it. It's been a while, but if I recall correctly, the optimized, stripped exe's at the time were a bit under 100mb. >500mb with debug symbols, etc.
- hydroreadsstuff 7y agoThe complexity of current hardware is madness. The performance of software is influenced by the workings of tens of units (+local caches) connected (non-linearly) with fifos, ooo buffers, replay mechanism. There are multiple versions of the same unit to save chip space (like light and heavy Integer/FP). Units/domains have different clock speeds. And from generation to generation port connections and instructions between pipelines can be reshuffled. I suppose what keeps performance changes relatively straightforward for developers is the set of benchmarks used to evaluate the hardware early on. I appreciate the article focusing on the I-Cache, and the nice intro to decoding. I would have preferred having an example, and improving something instead of abstractly talking about problems, effects of code and optimizations and possible workarounds. Tangent: I wonder if we will be seeing specialized instructions sometime that span multiple units and multiple cores in order to reduce data-movement. Think matrix-matrix multiplication. The potential improvements for power and speed seem huge.
- tempguy9999 7y ago> There are multiple versions of the same unit to save chip space (like light and heavy Integer/FP). I've literally never heard of this. I know you can have multiple execution units for parallel instruction execution, but 'light' and 'heavy' - can you give some info or a link? TIA
- hydroreadsstuff 7y agohttps://images.anandtech.com/doci/13699/Ronak26.jpg https://images.anandtech.com/doci/13699/Ronak26.jpg https://www.anandtech.com/show/13699/intel-architecture-day-2018-core-future-hybrid-x86/2 https://www.anandtech.com/show/13699/intel-architecture-day-... I believe this shows differences between FP and Integer units. In order to achieve a certain performance goal you don't necessarily need another integer divider when you want a new adder/multiplier. So you add a slimmer unit instead. I listed this in my original comment, because this is a giant can of worms for the compiler and decision-maker on where to execute what.
- tempguy9999 7y ago
- shereadsthenews 7y agoI wish the article discussed huge pages for executable text and their beneficial impact on the iTLB.
- htfy96 7y agoAnother big issue is iTLB misses. Today if you profile a real-life application, the iTLB miss rates can be 5%+ due to increased program image size. Splitting hot/cold variant like this article also works for this problem, but I was wondering if we can use transparent huge page on code while disabling it for mmapped data pages