5 ms·

On the contrary, I've often seen the "you can't beat the compiler" statement. This[1] recent reddit thread has it in the top comment, which is what prompted me

by kayamon 10y ago

On the contrary, I've often seen the "you can't beat the compiler" statement. This[1] recent reddit thread has it in the top comment, which is what prompted me to test it out. And while all those other points are fine points (and I mention all that in the conclusion), it doesn't change the fact that beating the compiler isn't always the rocket science it's made out to be. [1] https://www.reddit.com/r/programming/comments/5f9evm/learning_to_read_x86_assembly_language/ https://www.reddit.com/r/programming/comments/5f9evm/learnin...

imauld 10y ago
Are you the author of the linked post? If so I have a couple of questions: - Why not throw out the best and worst cases for each and then find the mean of run times? Seems like a more "fair" way to compare them. - Did you compare the assembly generated by the compiler to the assembly you wrote?
- Vendan 10y ago
  You should always be comparing best case for this kind of thing. Slower cases are most likely "your thread got switched out by the OS to let something else run", and that's not really a fair test.
  - qznc 10y ago
    If you want to guard against context switches by the OS, don't use stuff that "most likely" works. Measure with perf and let it count the context switches.
  - zamalek 10y ago
    Which is why you use 90th percentile.
    - to3m 10y ago
      What's wrong with using the best result? If you're concerned the code could run faster than possible: don't be :)
      - bArray 10y ago
        What if the test data is random? Could just get lucky and get a happy day scenario.
        Vendan 10y ago
        Then both versions should get that "happy day"? If you are using distinct random data for each version, then you aren't really benchmarking properly.
        bArray 10y ago
        It doesn't say that they both use the same data sets.
        Vendan 10y ago
        If they are using different data sets, then I'd say it's an invalid benchmark.
- panic 10y ago
  Your first question was asked and answered here: https://news.ycombinator.com/item?id=13060404 https://news.ycombinator.com/item?id=13060404
- emeryberger 10y ago
  A note: every compiler writer should read this paper. https://emeryberger.com/research/stabilizer/ https://emeryberger.com/research/stabilizer/
Joky 10y ago
Fair. As a compiler engineer (full-time on clang/LLVM), I wouldn't make such ridiculous claim (even though the compiler is capable of nasty tricks that "normal" humans wouldn't be able to pull). Someone that pretends that it is not possible to beat the compiler should start by taking something like a GEMM routine (for example from there https://github.com/flame/blis/tree/master/kernels/x86_64 https://github.com/flame/blis/tree/master/kernels/x86_64 ) and reimplement it to show how a mainstream optimizing compiler can do better using C/C++ or Fortran. A good starting point to understand the gap between hand-written optimized assembly and what you can get with C is http://apfel.mathematik.uni-ulm.de/~lehn/sghpc/gemm/index.html http://apfel.mathematik.uni-ulm.de/~lehn/sghpc/gemm/index.ht...

by kayamon 10y ago

imauld 10y ago

Are you the author of the linked post? If so I have a couple of questions: - Why not throw out the best and worst cases for each and then find the mean of run times? Seems like a more "fair" way to compare them. - Did you compare the assembly generated by the compiler to the assembly you wrote?

Vendan 10y ago
You should always be comparing best case for this kind of thing. Slower cases are most likely "your thread got switched out by the OS to let something else run", and that's not really a fair test.
- qznc 10y ago
  If you want to guard against context switches by the OS, don't use stuff that "most likely" works. Measure with perf and let it count the context switches.
- zamalek 10y ago
  Which is why you use 90th percentile.
  - to3m 10y ago
    What's wrong with using the best result? If you're concerned the code could run faster than possible: don't be :)
    - bArray 10y ago
      What if the test data is random? Could just get lucky and get a happy day scenario.
      - Vendan 10y ago
        Then both versions should get that "happy day"? If you are using distinct random data for each version, then you aren't really benchmarking properly.
        bArray 10y ago
        It doesn't say that they both use the same data sets.
        Vendan 10y ago
        If they are using different data sets, then I'd say it's an invalid benchmark.
panic 10y ago
Your first question was asked and answered here: https://news.ycombinator.com/item?id=13060404 https://news.ycombinator.com/item?id=13060404
emeryberger 10y ago
A note: every compiler writer should read this paper. https://emeryberger.com/research/stabilizer/ https://emeryberger.com/research/stabilizer/

Joky 10y ago

Fair. As a compiler engineer (full-time on clang/LLVM), I wouldn't make such ridiculous claim (even though the compiler is capable of nasty tricks that "normal" humans wouldn't be able to pull). Someone that pretends that it is not possible to beat the compiler should start by taking something like a GEMM routine (for example from there https://github.com/flame/blis/tree/master/kernels/x86_64 https://github.com/flame/blis/tree/master/kernels/x86_64 ) and reimplement it to show how a mainstream optimizing compiler can do better using C/C++ or Fortran. A good starting point to understand the gap between hand-written optimized assembly and what you can get with C is http://apfel.mathematik.uni-ulm.de/~lehn/sghpc/gemm/index.html http://apfel.mathematik.uni-ulm.de/~lehn/sghpc/gemm/index.ht...