4 ms·

> I have another (in this case, Rust) program that demonstrates 2x performance difference based on the order in which exactly two instructions are emitted. ...

> I have another (in this case, Rust) program that demonstrates 2x performance difference based on the order in which exactly two instructions are emitted. ... That loop dominated execution time, for an almost 2x overall result. I think a program like this—where two instructions in one loop is almost the entire execution time—isn't the right one to expect profile-guided optimization to greatly improve. I think instead it's complex programs where there's too much hot stuff to reasonably put human attention into it all. On programs like this, I really have gotten 15% improvements from an AutoFDO pipeline. Honestly I don't know exactly how it achieves that. I can make some guesses but will probably be wrong. Eg I was surprised when the "machine function splitter" was introduced recently that it wasn't already doing that. But generally speaking, I'd expect skilled human optimization to make the most difference at the smallest scales: less hot code, changing infrequently, targeting fewer critical (micro)architectures. I'd expect compiler optimization to make the most difference elsewhere. I don't think profile-guided optimization is an exception to this general rule. > To choose correctly depends on the compiler knowing a great deal more about the target microarchitecture than can reasonably be expected of a compiler. fwiw, it doesn't seem crazy to me to expect the compiler to know what operations can be fused.

ncmncm 6y ago
I explained the point in the very next paragraph, starting "The point I make...". Sometimes the compiler can guess what operations can be fused, if it knows what hardware the program will run on, and the vendor has revealed those details, and the compiler has incorporated all of them. But almost all programs are built for an ancient target, using a compiler version older than the actual target. When you really care about performance of a program, you compile it yourself, if you can, and add as much "-march=", "-lto", etc. hackery as you trust, and measure what difference all the different combinations (which interact!) make, and make sure they didn't introduce bugs (which they often do). But, as I noted, even after that there are lots of 2x effects the compiler doesn't understand. Many programs don't spend all their time in one loop, but almost all spend all their time in loops.
- scottlamb 6y ago
  Right, everything you've said is consistent with my "I'd expect skilled human optimization to make the most difference at the smallest scales: less hot code, changing infrequently, targeting fewer critical (micro)architectures". In other situations, many folks (including me) get 15% improvements from PGO, and it's hard to take seriously any argument that suggests that can't happen.
  - ncmncm 6y ago
    In other words, you didn't read what you were replying to: "That [optimizers] achieve any positive result, on average, with or without profile guidance, is impressive". What you call "skilled human optimization" appears to refer to the noted more or less accidental optimizations cited. We may congratulate ourselves when we find one that turns out can be applied elsewhere, and even program it into our compiler, but these are benefits more often of luck than skill, and are often much less durable than we hoped, washed away in the next microarchitectural generation, and sometimes even come to degrade performance. In the second example cited, recent Gcc versions deliberately intersperse other instructions between those that could be fused, to fill in for pipeline delays in a previous generation; but the benchmarks referenced for current chips were compiled with a previous compiler that didn't. So, current compilers improve performance for obsolete hardware and pessimize current hardware. Potential improvements in the next microarchitectural generation are often held hostage by optimizations performed in old compilers, used in reference benchmarks, that would be pessimizations in the better microarchitecture (therefore) not shipped.
    - scottlamb 6y ago
      You led with "my experiments with profile-guided optimization have been thus far disappointing". My point is that in many cases people get 15% improvements. Do you consider that a disappointing result? Are you disregarding other people's results? or are you disappointed that they don't apply in your specific situations? I originally gave you the benefit of the doubt by thinking it was the last of these, and speculated about how your situation differs. Now I don't know what you want, except to show off how smart you are vs how poor a reader I am, regardless of accuracy. I'm not interested. Bye.

ncmncm 6y ago

I explained the point in the very next paragraph, starting "The point I make...". Sometimes the compiler can guess what operations can be fused, if it knows what hardware the program will run on, and the vendor has revealed those details, and the compiler has incorporated all of them. But almost all programs are built for an ancient target, using a compiler version older than the actual target. When you really care about performance of a program, you compile it yourself, if you can, and add as much "-march=", "-lto", etc. hackery as you trust, and measure what difference all the different combinations (which interact!) make, and make sure they didn't introduce bugs (which they often do). But, as I noted, even after that there are lots of 2x effects the compiler doesn't understand. Many programs don't spend all their time in one loop, but almost all spend all their time in loops.

scottlamb 6y ago
Right, everything you've said is consistent with my "I'd expect skilled human optimization to make the most difference at the smallest scales: less hot code, changing infrequently, targeting fewer critical (micro)architectures". In other situations, many folks (including me) get 15% improvements from PGO, and it's hard to take seriously any argument that suggests that can't happen.
- ncmncm 6y ago
  In other words, you didn't read what you were replying to: "That [optimizers] achieve any positive result, on average, with or without profile guidance, is impressive". What you call "skilled human optimization" appears to refer to the noted more or less accidental optimizations cited. We may congratulate ourselves when we find one that turns out can be applied elsewhere, and even program it into our compiler, but these are benefits more often of luck than skill, and are often much less durable than we hoped, washed away in the next microarchitectural generation, and sometimes even come to degrade performance. In the second example cited, recent Gcc versions deliberately intersperse other instructions between those that could be fused, to fill in for pipeline delays in a previous generation; but the benchmarks referenced for current chips were compiled with a previous compiler that didn't. So, current compilers improve performance for obsolete hardware and pessimize current hardware. Potential improvements in the next microarchitectural generation are often held hostage by optimizations performed in old compilers, used in reference benchmarks, that would be pessimizations in the better microarchitecture (therefore) not shipped.
  - scottlamb 6y ago
    You led with "my experiments with profile-guided optimization have been thus far disappointing". My point is that in many cases people get 15% improvements. Do you consider that a disappointing result? Are you disregarding other people's results? or are you disappointed that they don't apply in your specific situations? I originally gave you the benefit of the doubt by thinking it was the last of these, and speculated about how your situation differs. Now I don't know what you want, except to show off how smart you are vs how poor a reader I am, regardless of accuracy. I'm not interested. Bye.