Instead of wall time, the cycle counts should be a little less noisy as they will actually account for the threads getting switched out. In this case they show

Instead of wall time, the cycle counts should be a little less noisy as they will actually account for the threads getting switched out. In this case they show a relatively similar reduction from a quick glance. > One plausible explanation would be that PGO improves instruction cache utilization, something which makes a difference for execution time but would not be reflected in the amount of instructions executed. Yes, this is a big reason why PGO is useful. It places the hot code together for improved ICache performance, while keeping the colder basic blocks separate (though there are likely some strategies here I don't know about). You could show the ICache miss counter and prove that this is the case. > I also don't know how branch mispredictions factor into instruction counts -- branch prediction being another aspect explicitly targeted by PGO. Branch mispredictions won't show up directly in the instruction counts, but they will show up indirectly through lower CPI or you can explicitly look at the counters for branch misses and compare between the two runs. If you were asking about whether speculative instructions from the missed branch are counted, they are not. Instructions are only counted once they are retired. (Intels counter for this is specifically named "Instructions Retired" to clarify that distinction). > In order to make the collected data as useful as possible, we should try to exercise all the common code paths within the compiler. I typically use the "standard" rustc-perf benchmark suite for this purpose This is not correct, though maybe just misstated. You need to exercise the code with inputs that are representative of the code you want to optimize for. Mozilla does this with the Firefox build (presumably) by using the Firefox build itself as an input since it is small enough. That's a perfect 1:1, so their performance gains are as good as possible. This post does the same thing, using the same Benchmark suite as training data and for testing, but the goal is to use this optimized rustc to compile other Rust programs. It would probably be better to show the results on a separate set of benchmarks to see how well PGO does for the general use case of "compiling Rust code" instead of "compiling the training data". Source: Did a lot of performance analysis (with a heavy use of counters) at my last job at Microsoft. They were very useful when we needed to do detailed analysis between x86 and ARM, among other things. I didn't work on PGO but a sibling team did, it was used extensively for the Windows builds. It was an interesting job, but unfortunately moving further up the stack to be a low level code monkey working on boring backend development at a colorful cloud company pays significantly better.