3 ms·

It’s interesting to me how frequently people will talk about sampling performance profilers and not know about Nyquist. Given a CPU runs at many GHz but SW sam

It’s interesting to me how frequently people will talk about sampling performance profilers and not know about Nyquist. Given a CPU runs at many GHz but SW sampling profilers run at ~1 or even maximum 10khz, it’s really hard to write software if you’re targeting processing at MHz rates.

rcxdude 3y ago
I'm not sure what you mean here, but I would say that sampling profilers are a good example of Nyquist applied correctly for the most part: your signal (the distribution of where your software is spending its time) is likely to have an extremely low bandwidth (as in, it's basically static), so why try to sample super rapidly? It's much more of a question of whether you get enough samples to represent that distribution accurately, not how quickly you get those samples.
- anonymous-panda 3y ago
  You’re talking about profiling a normal running program. If you’re profiling a benchmark that’s always executing the hot path, where exactly the hot path contribution lies becomes less clear. This is because you run into aliasing with the samples being collected at unhelpful points. Maybe if you run long enough you start to see a picture but at 1khz you’re going to have to run for a very long time. The other way aliasing comes into effect is that it becomes to see the impact of “individually cheap” but often executed pieces of code (eg a simple bounds check may not show up even if it’s responsible for a 20% slowdown because the probability of your sampler hitting it is small when that bounds check takes nanoseconds to execute vs your ms profiler sample rate. Basically a 1ms sampler can pick out a signal that’s 2ms or longer in periodicity if sampled once (all faster signals will get aliased). To get to 1ghz (once a nanosecond) would require capturing 1 million times more samples and you’re still dealing with aliasing screwing up the picture you’re getting (although maybe with stack sampling you get disambiguation to combat aliasing? Not sure).
  - rcxdude 3y ago
    I don't think what you're talking about is aliasing: it's more to do with the statistics of sampling. Though even then I still don't quite get what you mean: if a bounds check is 20% of your runtime then you're going to see it in your samples pretty quickly. If it's a small enough fraction of your runtime that you don't expect to see it in millions of samples, then why is it relevant to your performance? Now, if you're worried about latency outliers, I can see why sampling may not be a useful tool, but again I don't think the reason for that is really aliasing.
    - anonymous-panda 3y ago
      If the 20% is a hotspot yes. If the 20% is because it’s been inlined and split across 100 different call sites each contributing 0.2%, I don’t think it’s so easy to spot.
      - rcxdude 3y ago
        Sure, but for basically the same reason that it's difficult to spot in a trace: you need to classify all those segments of code as the same thing.
  - bigbillheck 3y ago
    > a simple bounds check may not show up even if it’s responsible for a 20% slowdown because the probability of your sampler hitting it is small when that bounds check takes nanoseconds to execute vs your ms profiler sample rate. Surely for this to happen you'd have to be putting a lot of effort into getting a perfect 1ms sampling rate, and even a little bit of variation in that would be more than enough to handle aliasing issues.
    - anonymous-panda 3y ago
      It’s been a few years but if I recall correctly the fact that there’s variation in the sampling itself makes the aliasing worse not better. At the very least should be no different.
- gct 3y ago
  Fun fact static is very broadband

rcxdude 3y ago

I'm not sure what you mean here, but I would say that sampling profilers are a good example of Nyquist applied correctly for the most part: your signal (the distribution of where your software is spending its time) is likely to have an extremely low bandwidth (as in, it's basically static), so why try to sample super rapidly? It's much more of a question of whether you get enough samples to represent that distribution accurately, not how quickly you get those samples.

anonymous-panda 3y ago
You’re talking about profiling a normal running program. If you’re profiling a benchmark that’s always executing the hot path, where exactly the hot path contribution lies becomes less clear. This is because you run into aliasing with the samples being collected at unhelpful points. Maybe if you run long enough you start to see a picture but at 1khz you’re going to have to run for a very long time. The other way aliasing comes into effect is that it becomes to see the impact of “individually cheap” but often executed pieces of code (eg a simple bounds check may not show up even if it’s responsible for a 20% slowdown because the probability of your sampler hitting it is small when that bounds check takes nanoseconds to execute vs your ms profiler sample rate. Basically a 1ms sampler can pick out a signal that’s 2ms or longer in periodicity if sampled once (all faster signals will get aliased). To get to 1ghz (once a nanosecond) would require capturing 1 million times more samples and you’re still dealing with aliasing screwing up the picture you’re getting (although maybe with stack sampling you get disambiguation to combat aliasing? Not sure).
- rcxdude 3y ago
  I don't think what you're talking about is aliasing: it's more to do with the statistics of sampling. Though even then I still don't quite get what you mean: if a bounds check is 20% of your runtime then you're going to see it in your samples pretty quickly. If it's a small enough fraction of your runtime that you don't expect to see it in millions of samples, then why is it relevant to your performance? Now, if you're worried about latency outliers, I can see why sampling may not be a useful tool, but again I don't think the reason for that is really aliasing.
  - anonymous-panda 3y ago
    If the 20% is a hotspot yes. If the 20% is because it’s been inlined and split across 100 different call sites each contributing 0.2%, I don’t think it’s so easy to spot.
    - rcxdude 3y ago
      Sure, but for basically the same reason that it's difficult to spot in a trace: you need to classify all those segments of code as the same thing.
- bigbillheck 3y ago
  > a simple bounds check may not show up even if it’s responsible for a 20% slowdown because the probability of your sampler hitting it is small when that bounds check takes nanoseconds to execute vs your ms profiler sample rate. Surely for this to happen you'd have to be putting a lot of effort into getting a perfect 1ms sampling rate, and even a little bit of variation in that would be more than enough to handle aliasing issues.
  - anonymous-panda 3y ago
    It’s been a few years but if I recall correctly the fact that there’s variation in the sampling itself makes the aliasing worse not better. At the very least should be no different.
gct 3y ago
Fun fact static is very broadband