3 ms·

GCC is part of spec2006, which they optimize for. But the actual compilers people use (IE not that version of GCC) are rarely well-optimized for CPU. In fact,

GCC is part of spec2006, which they optimize for. But the actual compilers people use (IE not that version of GCC) are rarely well-optimized for CPU. In fact, most compiler profiles are not flat yet, and where your compilation takes a long time, it's because some optimization or algorithm has gone nuts and needs to be fixed. Until that kind of state of the world changes, measuring compile time is IMHO, a bit pointless, because they are only artificially CPU bound, and not in a way that is useful to benchmark (IE while i understand that people use these compilers every day, and care about performance, i'm just saying that, given a compiler benchmark, i could still probably make it take zero time with limited amounts of work)

rockdoe 10y ago
I think you're on the slippery slope here that lead to thinking Itanium was a good idea.
brandmeyer 10y ago
GCC is a useful benchmark not because it is unoptimized, but because it represents a real-world work profile that is otherwise poorly represented in SPEC. The working set sizes are large relative to the cache size, and its composed of a bunch of pointer-chasing branch-heavy logic as it winds through the various trees and intermediate languages. That profile is quite a bit closer to the profile of common business and server applications than nearly all of the other members in SPEC.
- DannyBee 10y ago
  "GCC is a useful benchmark not because it is unoptimized, but because it represents a real-world work profile that is otherwise poorly represented in SPEC." Sure, in that sense, but you also don't want benchmarks that are trivial to game by performing optimizations to them. :)
  - brandmeyer 10y ago
    For SPEC, you can't edit the source code of the unit under test. Only the compiler and host hardware count. Some components of SPEC have been "broken" by compilers in the past, by implementing some optimization or other that suddenly made that particular test code much faster. But you don't ever get to re-write the benchmark itself, you can't replace a 2^N path with an N^3 path or anything like that. So these kinds of breakages have almost always been restricted to benchmarks that are reducible to one or two relatively tight inner arithmetic-heavy loop(s). If someone managed to make an optimization that could suddenly make GCC 2x faster without humans editing GCC's source, that would be a major CS breakthrough :)
    - DannyBee 10y ago
      I'm aware of the limitations on SPEC editing. However, "If someone managed to make an optimization that could suddenly make GCC 2x faster without humans editing GCC's source, that would be a major CS breakthrough :) " No, it really wouldn't. For example, if you optimize the line-ending and tokenization finding in GCC's parser to use SIMD, you would become significantly faster. Not 2x, but significantly. These are entirely doable by compilers. " But you don't ever get to re-write the benchmark itself, you can't replace a 2^N path with an N^3 path or anything like that." Except, you can, by hoisting calls out of loops, etc. There are also dynamic memoization optimziations that can be applied, etc. "If someone managed to make an optimization " This is a common fallacy that it's ever really one optimization. It's usually combinations of optimizations, each buying you N%. I will state outright, having worked on pretty much all of the slow parts of gcc (according to spec profiles), and having a team of 80+ people working on compilers, 2x on gcc through compiler optimization is more than within the realm of possibility.
imtringued 10y ago
>In fact, most compiler profiles are not flat yet, and where your compilation takes a long time, it's because some optimization or algorithm has gone nuts and needs to be fixed. For some reason one of my sideprojects needs more than two minutes to link but the task manager tells me it's neither CPU nor RAM nor disk bound. I haven't found the cause yet.

rockdoe 10y ago

I think you're on the slippery slope here that lead to thinking Itanium was a good idea.

brandmeyer 10y ago

GCC is a useful benchmark not because it is unoptimized, but because it represents a real-world work profile that is otherwise poorly represented in SPEC. The working set sizes are large relative to the cache size, and its composed of a bunch of pointer-chasing branch-heavy logic as it winds through the various trees and intermediate languages. That profile is quite a bit closer to the profile of common business and server applications than nearly all of the other members in SPEC.

DannyBee 10y ago
"GCC is a useful benchmark not because it is unoptimized, but because it represents a real-world work profile that is otherwise poorly represented in SPEC." Sure, in that sense, but you also don't want benchmarks that are trivial to game by performing optimizations to them. :)
- brandmeyer 10y ago
  For SPEC, you can't edit the source code of the unit under test. Only the compiler and host hardware count. Some components of SPEC have been "broken" by compilers in the past, by implementing some optimization or other that suddenly made that particular test code much faster. But you don't ever get to re-write the benchmark itself, you can't replace a 2^N path with an N^3 path or anything like that. So these kinds of breakages have almost always been restricted to benchmarks that are reducible to one or two relatively tight inner arithmetic-heavy loop(s). If someone managed to make an optimization that could suddenly make GCC 2x faster without humans editing GCC's source, that would be a major CS breakthrough :)
  - DannyBee 10y ago
    I'm aware of the limitations on SPEC editing. However, "If someone managed to make an optimization that could suddenly make GCC 2x faster without humans editing GCC's source, that would be a major CS breakthrough :) " No, it really wouldn't. For example, if you optimize the line-ending and tokenization finding in GCC's parser to use SIMD, you would become significantly faster. Not 2x, but significantly. These are entirely doable by compilers. " But you don't ever get to re-write the benchmark itself, you can't replace a 2^N path with an N^3 path or anything like that." Except, you can, by hoisting calls out of loops, etc. There are also dynamic memoization optimziations that can be applied, etc. "If someone managed to make an optimization " This is a common fallacy that it's ever really one optimization. It's usually combinations of optimizations, each buying you N%. I will state outright, having worked on pretty much all of the slow parts of gcc (according to spec profiles), and having a team of 80+ people working on compilers, 2x on gcc through compiler optimization is more than within the realm of possibility.

imtringued 10y ago

>In fact, most compiler profiles are not flat yet, and where your compilation takes a long time, it's because some optimization or algorithm has gone nuts and needs to be fixed. For some reason one of my sideprojects needs more than two minutes to link but the task manager tells me it's neither CPU nor RAM nor disk bound. I haven't found the cause yet.