3 ms·

Some thorough benchmarks are something I'd love to read about here on HN. If you ever do those benchmarks, post 'em.

by pxc 4y ago

bravetraveler 4y ago
I'll be happy to share! I'll probably be waiting for the next release so I can get their pointer-enabled builds. However they do that may give me helpful hints RE: rebuilding the same sources without it. I know the tools to do it, but what I'm imagining right now is a bit convoluted. I'm open to suggestions on things to test. I want to avoid too much reliance on synthetics -- and my 'real world' differs like all of ours My original post has (currently) been down-struck a bit but you're the only reply. I appreciate you taking the time to engage I understand it seems dismissive, perhaps even aggressive, but the Fedora/SuSE people tend to set trends. I just want to measure them. The snark is me finding it ironic that I can (relatively easily) measure this with the same methods they want to avoid... rebuilds with the right options under mock / fedpkg subtext: not that many people find profiling interesting or important... but they will if you start messing with their performance. I find it interesting though so I'm not mad, I just fairly called it self-fulfilling -- I'm going to be profiling because the minority who regularly do, don't want to rebuild (like I will be)
evmar 4y ago
https://docs.kernel.org/x86/orc-unwinder.html https://docs.kernel.org/x86/orc-unwinder.html (which I found when looking up ORC, as mentioned in this thread) has some stats for the Linux kernel on the cost of frame pointers: "5-10%" and "3.49% more cache misses with framepointer enabled and 6.59% more cycles". Between this, the various sorts of C language safety checks compilers have been slowly adding, and all the spectre-related defenses, it really does feel like performance is getting slowly consumed by a series of single-digit percentage tweaks.
- mepian 4y ago
  It's just the time to pay the piper finally arriving, the computer industry ignored security in favor of performance for far too long.
  - nullc 4y ago
    > the computer industry ignored security in favor of performance for far too long Doesn't including an explicit, predictable, frame pointer make life moderately easier for shell code?
- bravetraveler 4y ago
  I agree -- it's easy to think that a few percent here/there don't hurt, but there's no global context on this. Team A takes away ~4% here, B goes on to do ~3%, and so on and so forth. In only five of these, and consider the chains of Pythonic imports you've seen for example, you've lost roughly a quarter of your performance. (grant me an additive/compounding? nature) This is assuming that the measurements taken in a vacuum even hold up to reality. I've seen reports for misc. other things in the 20% range, so my confidence is remarkably low. I'm not against making it easier, but I don't think everyone always need to be ready for profiling. Making opting in easier lets us ignore the hand-wavy numbers. edit: To add, the simplification of this performance cost down to percentages completely ignores scale. It's a deceptive statistician's trick. Your favorite browser accidentally releasing a debug/verbose-logging version that consumed in the range of 1-10% more power globally would be significant.
  - ilyt 4y ago
    And each 2-3% reduction is reason that the following 2-3% reduction asks "the previous one was accepted, why not accept this one?"

Some thorough benchmarks are something I'd love to read about here on HN. If you ever do those benchmarks, post 'em.

by pxc 4y ago

Some thorough benchmarks are something I'd love to read about here on HN. If you ever do those benchmarks, post 'em.

I'll be happy to share! I'll probably be waiting for the next release so I can get their pointer-enabled builds. However they do that may give me helpful hints RE: rebuilding the same sources without it. I know the tools to do it, but what I'm imagining right now is a bit convoluted. I'm open to suggestions on things to test. I want to avoid too much reliance on synthetics -- and my 'real world' differs like all of ours My original post has (currently) been down-struck a bit but you're the only reply. I appreciate you taking the time to engage I understand it seems dismissive, perhaps even aggressive, but the Fedora/SuSE people tend to set trends. I just want to measure them. The snark is me finding it ironic that I can (relatively easily) measure this with the same methods they want to avoid... rebuilds with the right options under mock / fedpkg subtext: not that many people find profiling interesting or important... but they will if you start messing with their performance. I find it interesting though so I'm not mad, I just fairly called it self-fulfilling -- I'm going to be profiling because the minority who regularly do, don't want to rebuild (like I will be)

evmar 4y ago

https://docs.kernel.org/x86/orc-unwinder.html https://docs.kernel.org/x86/orc-unwinder.html (which I found when looking up ORC, as mentioned in this thread) has some stats for the Linux kernel on the cost of frame pointers: "5-10%" and "3.49% more cache misses with framepointer enabled and 6.59% more cycles". Between this, the various sorts of C language safety checks compilers have been slowly adding, and all the spectre-related defenses, it really does feel like performance is getting slowly consumed by a series of single-digit percentage tweaks.

mepian 4y ago
It's just the time to pay the piper finally arriving, the computer industry ignored security in favor of performance for far too long.
- nullc 4y ago
  > the computer industry ignored security in favor of performance for far too long Doesn't including an explicit, predictable, frame pointer make life moderately easier for shell code?
bravetraveler 4y ago
I agree -- it's easy to think that a few percent here/there don't hurt, but there's no global context on this. Team A takes away ~4% here, B goes on to do ~3%, and so on and so forth. In only five of these, and consider the chains of Pythonic imports you've seen for example, you've lost roughly a quarter of your performance. (grant me an additive/compounding? nature) This is assuming that the measurements taken in a vacuum even hold up to reality. I've seen reports for misc. other things in the 20% range, so my confidence is remarkably low. I'm not against making it easier, but I don't think everyone always need to be ready for profiling. Making opting in easier lets us ignore the hand-wavy numbers. edit: To add, the simplification of this performance cost down to percentages completely ignores scale. It's a deceptive statistician's trick. Your favorite browser accidentally releasing a debug/verbose-logging version that consumed in the range of 1-10% more power globally would be significant.
- ilyt 4y ago
  And each 2-3% reduction is reason that the following 2-3% reduction asks "the previous one was accepted, why not accept this one?"