3 ms·

What techniques do people use to profile code where the core path takes <<1ms? A sampling profiler is max 1khz and thus aliasing artifacts will hide the true so

What techniques do people use to profile code where the core path takes <<1ms? A sampling profiler is max 1khz and thus aliasing artifacts will hide the true source of hotspots I find and valgrind is too slow. The other problem I have is that diffuse hotspots are hard to root cause when running the optimized version (eg hard to spot bounds checking). I’m not confident that simply running the code under test longer helps with the aliasing artifacts (probably because if the core loop is running in 1us then I have to run for 1000x longer to remove the sampling bias due to Nyquist if that’s even the right strategy?)

alright2565 3y ago
I'm confused here. Does this core path only run once per second? I've had no problems profiling blocks of code that short, because it sounds like every inner loop to me. If you're looking to optimize a few hundred instructions, then focus on reading and understanding the code manually at the instruction level. A profiler will never help you there, CPUs are too complex: 1. I don't trust that the profiler has equal probability of pausing on any specific instruction 2. CPUs do no execute instructions in sequence, but in parallel. So you will need to understand data dependencies & maybe pipeline structures for your target processor. If you're getting really serious, you'll want something like https://uops.info/table.html https://uops.info/table.html
- vlovich123 3y ago
  So I’m toying with my own database implementation and I’m getting ~20M writes per second. The write operation is a critical path that takes 1/20Mhz (ie 50ns). A sampling profiler will only capture max 1000hz (and even that is pushing it). And a hot spot here is 5-10ns probably. And similarly it could very well be memory transfer costs vs cpu which I’ve found hard to distinguish. I’ve had luck with valgrind but just reading individual instructions has been uninformative as the code path is too complex for that kind of analysis.
phkahler 3y ago
>> What techniques do people use to profile code where the core path takes <<1ms? Make a loop that runs the code 5000 times and time that. What do you mean by "diffuse hot spots"? If a piece of code taking 1ms is called infrequently then optimizing it is not going to have a meaningful impact on your overall execution time. You can measure the time for longer running higher level functions that do a lot. If you then optimize some small function that's a leaf in the call graph, you can see the impact on the larger function - and it probably won't be much. One thing that can happen when nobody cares about performance is that every part of the code gets little inefficiencies and fixing any one of them has very little impact, but fixing ALL of them can be really significant. In that case you should start by profiling or timing the higher level functions and finding ones that have relatively small call trees that you can work on. Once you optimize one piece, the effects of optimizing others are a bit more significant because they are a larger part of overall execution time.
- vlovich123 3y ago
  Diffuse could be something like bounds checking in Rust. It’s not code I’ve written nor will any profiler call out the bounds check - they’ll at best point to the high level collection method but each one is a speed bump. Or the compiler generating inefficient code - spelunking through assembly hoping to spot the problem isn’t an effective use of time and profilers I’ve tried (valgrind and hotspot) have been unhelpful in giving me the info. The advice you’ve provided is correct philosophy but not actually actionable - I’m having trouble finding the inefficiencies.
maccard 3y ago
A benchmark framework, e.g Google Benchmark, and then WPF/Superluminal/Vtune/uProf.
artimaeis 3y ago
Profiling != benchmarking. Profiling is great to find _where_ you should setup benchmarks in your code. If you've got code in a path that consistently takes <1ms and isn't called enough times to significantly add to the profiling of the application, would it really warrant benchmark comparisons? As others have said, benchmark frameworks are great. Many systems have support for high-resolution perf counters these days. Just ran a quick test on my local system, it's able to make a timer accurate to 100ns. Benchmark frameworks let you hook into your code in such a way that you can easily run a single fragment 10k-10m times and produce an analysis of the result that includes mean time, std deviation, and memory allocations. They usually let you setup a side by side comparison so you can directly compare 2+ methods. If you use a language with a JIT then good frameworks for those include a warmup period to get the best-JIT version of the code in place before collecting data.
- vlovich123 3y ago
  Yes I’ve done all that. But I’m still at being able to do the operation 20M times/s (ie 50ns) whereas a prototype I tested could do 100M (10ns) so I’m having trouble spotting the speed bumps that are in the way.
charleshn 3y ago
I really like magic-trace [0]. https://github.com/janestreet/magic-trace https://github.com/janestreet/magic-trace Not that the exact tracing relies on Intel PT - support for AMD was added recently but uses perf so suffers from the same sampling/skew issues, but is still very useful.
- vlovich123 3y ago
  Thanks for the tip. I thankfully went with Intel so it should work and looks like it might get close. My needs are a bit higher frequency than that but it should help with other use cases that aren’t as high frequency + maybe there’s a way to set it to only sample instrumented functions so that the timer cost is further amortized.
Veserv 3y ago
You can use function entry/exit (FEE) tracing. A optimized implementation should only incur overhead around one hardware timestamp per function call which is generally around 10-100 ns. That is sufficient for benchmarking quantitative differences at the 1000 ns level and even clear qualitative differences at the 100 ns level. The bias is also fairly stable so you can generally even subtract it out with a little bit of care. And really, the microarchitectural state starts dominating at that point anyways, so anything more fine-grained than that starts depending on making sure the surrounding code did not mess up your L2 or branch predictor and such.
- vlovich123 3y ago
  Do you have any link to something that does it? I’ve not been able to find anything. And the things I’m trying to optimize is 50ns -> 10ns so I’d need to be careful about the overhead but hopefully it can be set to a mode that only does it for instrumented functions I’m trying to profile rather than all entry/exit

alright2565 3y ago

I'm confused here. Does this core path only run once per second? I've had no problems profiling blocks of code that short, because it sounds like every inner loop to me. If you're looking to optimize a few hundred instructions, then focus on reading and understanding the code manually at the instruction level. A profiler will never help you there, CPUs are too complex: 1. I don't trust that the profiler has equal probability of pausing on any specific instruction 2. CPUs do no execute instructions in sequence, but in parallel. So you will need to understand data dependencies & maybe pipeline structures for your target processor. If you're getting really serious, you'll want something like https://uops.info/table.html https://uops.info/table.html

vlovich123 3y ago
So I’m toying with my own database implementation and I’m getting ~20M writes per second. The write operation is a critical path that takes 1/20Mhz (ie 50ns). A sampling profiler will only capture max 1000hz (and even that is pushing it). And a hot spot here is 5-10ns probably. And similarly it could very well be memory transfer costs vs cpu which I’ve found hard to distinguish. I’ve had luck with valgrind but just reading individual instructions has been uninformative as the code path is too complex for that kind of analysis.

phkahler 3y ago

>> What techniques do people use to profile code where the core path takes <<1ms? Make a loop that runs the code 5000 times and time that. What do you mean by "diffuse hot spots"? If a piece of code taking 1ms is called infrequently then optimizing it is not going to have a meaningful impact on your overall execution time. You can measure the time for longer running higher level functions that do a lot. If you then optimize some small function that's a leaf in the call graph, you can see the impact on the larger function - and it probably won't be much. One thing that can happen when nobody cares about performance is that every part of the code gets little inefficiencies and fixing any one of them has very little impact, but fixing ALL of them can be really significant. In that case you should start by profiling or timing the higher level functions and finding ones that have relatively small call trees that you can work on. Once you optimize one piece, the effects of optimizing others are a bit more significant because they are a larger part of overall execution time.

vlovich123 3y ago
Diffuse could be something like bounds checking in Rust. It’s not code I’ve written nor will any profiler call out the bounds check - they’ll at best point to the high level collection method but each one is a speed bump. Or the compiler generating inefficient code - spelunking through assembly hoping to spot the problem isn’t an effective use of time and profilers I’ve tried (valgrind and hotspot) have been unhelpful in giving me the info. The advice you’ve provided is correct philosophy but not actually actionable - I’m having trouble finding the inefficiencies.

maccard 3y ago

A benchmark framework, e.g Google Benchmark, and then WPF/Superluminal/Vtune/uProf.

artimaeis 3y ago

Profiling != benchmarking. Profiling is great to find _where_ you should setup benchmarks in your code. If you've got code in a path that consistently takes <1ms and isn't called enough times to significantly add to the profiling of the application, would it really warrant benchmark comparisons? As others have said, benchmark frameworks are great. Many systems have support for high-resolution perf counters these days. Just ran a quick test on my local system, it's able to make a timer accurate to 100ns. Benchmark frameworks let you hook into your code in such a way that you can easily run a single fragment 10k-10m times and produce an analysis of the result that includes mean time, std deviation, and memory allocations. They usually let you setup a side by side comparison so you can directly compare 2+ methods. If you use a language with a JIT then good frameworks for those include a warmup period to get the best-JIT version of the code in place before collecting data.

vlovich123 3y ago
Yes I’ve done all that. But I’m still at being able to do the operation 20M times/s (ie 50ns) whereas a prototype I tested could do 100M (10ns) so I’m having trouble spotting the speed bumps that are in the way.

charleshn 3y ago

I really like magic-trace [0]. https://github.com/janestreet/magic-trace https://github.com/janestreet/magic-trace Not that the exact tracing relies on Intel PT - support for AMD was added recently but uses perf so suffers from the same sampling/skew issues, but is still very useful.

vlovich123 3y ago
Thanks for the tip. I thankfully went with Intel so it should work and looks like it might get close. My needs are a bit higher frequency than that but it should help with other use cases that aren’t as high frequency + maybe there’s a way to set it to only sample instrumented functions so that the timer cost is further amortized.

Veserv 3y ago

You can use function entry/exit (FEE) tracing. A optimized implementation should only incur overhead around one hardware timestamp per function call which is generally around 10-100 ns. That is sufficient for benchmarking quantitative differences at the 1000 ns level and even clear qualitative differences at the 100 ns level. The bias is also fairly stable so you can generally even subtract it out with a little bit of care. And really, the microarchitectural state starts dominating at that point anyways, so anything more fine-grained than that starts depending on making sure the surrounding code did not mess up your L2 or branch predictor and such.

vlovich123 3y ago
Do you have any link to something that does it? I’ve not been able to find anything. And the things I’m trying to optimize is 50ns -> 10ns so I’d need to be careful about the overhead but hopefully it can be set to a mode that only does it for instrumented functions I’m trying to profile rather than all entry/exit