3 ms·
"SPEC has ruled that the compiler used for this result was performing a compilation that specifically improves the performance of the 523.xalancbmk_r / 623.xala
by almatabata 3y ago
"SPEC has ruled that the compiler used for this result was performing a compilation
that specifically improves the performance of the 523.xalancbmk_r / 623.xalancbmk_s
benchmarks using a priori knowledge of the SPEC code and dataset to perform a
transformation that has narrow applicability."
Does this mean that if I use their compiler in production I might get worse performance because they apply this optimization to my own code as well?
- riscy 3y agoIt’s probably designed to recognize a very specific code pattern in SPEC, so it’s very unlikely to affect any other code. If it wasn’t so hyper targeted to only kick in for those two benchmarks, then there’d be no issue: it’d just be a common (mis)optimization that happens to help SPEC.
- bastawhiz 3y agoMy read is that they do what Volkswagen did with the emissions scandal: they detect when they're compiling code for the benchmark, and compile it in a way that makes the benchmark code (specifically) faster. That optimization wouldn't generalize to "real" code because the optimization relies on the specific code and data ("data" being fixed non-code inputs processed at runtime) being processed by the benchmark to have an effect. If you know based on prior knowledge that, for instance, code that uses a signed integer will always only handle positive integers, you might cheat by optimizing it to be unsigned integers. But a compiler couldn't possibly know that in advance because it can't see the data the code processes at compile time. For your code to benefit, it would have to do the same things the benchmark is computing and have the same shape of data as the data included with the benchmark suite.