4 ms·

For SPEC, you can't edit the source code of the unit under test. Only the compiler and host hardware count. Some components of SPEC have been "broken" by comp

For SPEC, you can't edit the source code of the unit under test. Only the compiler and host hardware count. Some components of SPEC have been "broken" by compilers in the past, by implementing some optimization or other that suddenly made that particular test code much faster. But you don't ever get to re-write the benchmark itself, you can't replace a 2^N path with an N^3 path or anything like that. So these kinds of breakages have almost always been restricted to benchmarks that are reducible to one or two relatively tight inner arithmetic-heavy loop(s). If someone managed to make an optimization that could suddenly make GCC 2x faster without humans editing GCC's source, that would be a major CS breakthrough :)

DannyBee 10y ago
I'm aware of the limitations on SPEC editing. However, "If someone managed to make an optimization that could suddenly make GCC 2x faster without humans editing GCC's source, that would be a major CS breakthrough :) " No, it really wouldn't. For example, if you optimize the line-ending and tokenization finding in GCC's parser to use SIMD, you would become significantly faster. Not 2x, but significantly. These are entirely doable by compilers. " But you don't ever get to re-write the benchmark itself, you can't replace a 2^N path with an N^3 path or anything like that." Except, you can, by hoisting calls out of loops, etc. There are also dynamic memoization optimziations that can be applied, etc. "If someone managed to make an optimization " This is a common fallacy that it's ever really one optimization. It's usually combinations of optimizations, each buying you N%. I will state outright, having worked on pretty much all of the slow parts of gcc (according to spec profiles), and having a team of 80+ people working on compilers, 2x on gcc through compiler optimization is more than within the realm of possibility.

DannyBee 10y ago

I'm aware of the limitations on SPEC editing. However, "If someone managed to make an optimization that could suddenly make GCC 2x faster without humans editing GCC's source, that would be a major CS breakthrough :) " No, it really wouldn't. For example, if you optimize the line-ending and tokenization finding in GCC's parser to use SIMD, you would become significantly faster. Not 2x, but significantly. These are entirely doable by compilers. " But you don't ever get to re-write the benchmark itself, you can't replace a 2^N path with an N^3 path or anything like that." Except, you can, by hoisting calls out of loops, etc. There are also dynamic memoization optimziations that can be applied, etc. "If someone managed to make an optimization " This is a common fallacy that it's ever really one optimization. It's usually combinations of optimizations, each buying you N%. I will state outright, having worked on pretty much all of the slow parts of gcc (according to spec profiles), and having a team of 80+ people working on compilers, 2x on gcc through compiler optimization is more than within the realm of possibility.