5 ms·

Do they ever do compiler speed tests or boot speed tests? Which of the existing ones would be most similar?

by daxfohl 10y ago

simias 10y ago
Boot speed in my experience is mostly i/o bound so I doubt it makes a significant difference over a similar Intel chip. Is boot performance still an issue nowadays anyway? Since I've switched to SSDs all my computers boot up in a few seconds (Linux or Windows). And if you hibernate instead of shutting down it's even faster. Compiler tests sound more interesting to me, I'm looking forward to those benchmarks.
- baq 10y ago
  yeah kernel compilation, .net compilation, js linting, babeling etc. - those would be quite interesting.
- majewsky 10y ago
  Boot speed is not even IO-bound for me, it's crappy-firmware-bound. `systemd-analyze plot` shows a total boot time of ~15 seconds, of which ~10 seconds is spent on the firmware, ~3 seconds on the kernel and hardware initialization, and 1-2 seconds on the entire userland including desktop session.
- daxfohl 10y ago
  I'm interested in boot just for curiosity's sake. With the new PCIe SSDs, I wonder if CPU may actually be the bottleneck.
__voidptr__ 10y ago
Hardware.fr just published some compiler benchmarks: http://www.hardware.fr/articles/956-12/compilation-visual-studio-mingw-w64-gcc.html http://www.hardware.fr/articles/956-12/compilation-visual-st...
dvdgsng 10y ago
Regarding boot speed, here is quote from Computerbase: Boards initialization takes forever For the last weeks Computerbase had access to different mainboards [...]. They had an extremely long init process in common: even with the latest BIOS it took up to 30s to get to the POST screen. Obviously this needs optimization. ----- German: Die Boards initialisieren ewig ComputerBase konnte die letzte Woche auf drei verschiedene Mainboards zugreifen, neben den X370-Varianten von Asus und MSI auch auf das Gigabyte GA-AB350-Gaming 3 mit Chipsatz B350. Allen gemein war auch mit dem letzten BIOS der extrem langwierige Initialisierungsprozess: Bis der Post-Screen erscheint, vergeht über eine halbe Minute. Auch hier gibt es offensichtlich noch deutlichen Optimierungsbedarf. ---- https://www.computerbase.de/2017-03/amd-ryzen-1800x-1700x-1700-test/2/ https://www.computerbase.de/2017-03/amd-ryzen-1800x-1700x-17...
DannyBee 10y ago
GCC is part of spec2006, which they optimize for. But the actual compilers people use (IE not that version of GCC) are rarely well-optimized for CPU. In fact, most compiler profiles are not flat yet, and where your compilation takes a long time, it's because some optimization or algorithm has gone nuts and needs to be fixed. Until that kind of state of the world changes, measuring compile time is IMHO, a bit pointless, because they are only artificially CPU bound, and not in a way that is useful to benchmark (IE while i understand that people use these compilers every day, and care about performance, i'm just saying that, given a compiler benchmark, i could still probably make it take zero time with limited amounts of work)
- rockdoe 10y ago
  I think you're on the slippery slope here that lead to thinking Itanium was a good idea.
- brandmeyer 10y ago
  GCC is a useful benchmark not because it is unoptimized, but because it represents a real-world work profile that is otherwise poorly represented in SPEC. The working set sizes are large relative to the cache size, and its composed of a bunch of pointer-chasing branch-heavy logic as it winds through the various trees and intermediate languages. That profile is quite a bit closer to the profile of common business and server applications than nearly all of the other members in SPEC.
  - DannyBee 10y ago
    "GCC is a useful benchmark not because it is unoptimized, but because it represents a real-world work profile that is otherwise poorly represented in SPEC." Sure, in that sense, but you also don't want benchmarks that are trivial to game by performing optimizations to them. :)
    - brandmeyer 10y ago
      For SPEC, you can't edit the source code of the unit under test. Only the compiler and host hardware count. Some components of SPEC have been "broken" by compilers in the past, by implementing some optimization or other that suddenly made that particular test code much faster. But you don't ever get to re-write the benchmark itself, you can't replace a 2^N path with an N^3 path or anything like that. So these kinds of breakages have almost always been restricted to benchmarks that are reducible to one or two relatively tight inner arithmetic-heavy loop(s). If someone managed to make an optimization that could suddenly make GCC 2x faster without humans editing GCC's source, that would be a major CS breakthrough :)
stefantalpalaru 10y ago
Phoronix does a "timed Linux kernel compilation" test where the 1800X is king: http://www.phoronix.com/scan.php?page=article&item=ryzen-1800x-linux&num=5 http://www.phoronix.com/scan.php?page=article&item=ryzen-180...

Do they ever do compiler speed tests or boot speed tests? Which of the existing ones would be most similar?

by daxfohl 10y ago

Do they ever do compiler speed tests or boot speed tests? Which of the existing ones would be most similar?

Boot speed in my experience is mostly i/o bound so I doubt it makes a significant difference over a similar Intel chip. Is boot performance still an issue nowadays anyway? Since I've switched to SSDs all my computers boot up in a few seconds (Linux or Windows). And if you hibernate instead of shutting down it's even faster. Compiler tests sound more interesting to me, I'm looking forward to those benchmarks.

baq 10y ago
yeah kernel compilation, .net compilation, js linting, babeling etc. - those would be quite interesting.
majewsky 10y ago
Boot speed is not even IO-bound for me, it's crappy-firmware-bound. `systemd-analyze plot` shows a total boot time of ~15 seconds, of which ~10 seconds is spent on the firmware, ~3 seconds on the kernel and hardware initialization, and 1-2 seconds on the entire userland including desktop session.
daxfohl 10y ago
I'm interested in boot just for curiosity's sake. With the new PCIe SSDs, I wonder if CPU may actually be the bottleneck.

__voidptr__ 10y ago

Hardware.fr just published some compiler benchmarks: http://www.hardware.fr/articles/956-12/compilation-visual-studio-mingw-w64-gcc.html http://www.hardware.fr/articles/956-12/compilation-visual-st...

dvdgsng 10y ago

Regarding boot speed, here is quote from Computerbase: Boards initialization takes forever For the last weeks Computerbase had access to different mainboards [...]. They had an extremely long init process in common: even with the latest BIOS it took up to 30s to get to the POST screen. Obviously this needs optimization. ----- German: Die Boards initialisieren ewig ComputerBase konnte die letzte Woche auf drei verschiedene Mainboards zugreifen, neben den X370-Varianten von Asus und MSI auch auf das Gigabyte GA-AB350-Gaming 3 mit Chipsatz B350. Allen gemein war auch mit dem letzten BIOS der extrem langwierige Initialisierungsprozess: Bis der Post-Screen erscheint, vergeht über eine halbe Minute. Auch hier gibt es offensichtlich noch deutlichen Optimierungsbedarf. ---- https://www.computerbase.de/2017-03/amd-ryzen-1800x-1700x-1700-test/2/ https://www.computerbase.de/2017-03/amd-ryzen-1800x-1700x-17...

DannyBee 10y ago

GCC is part of spec2006, which they optimize for. But the actual compilers people use (IE not that version of GCC) are rarely well-optimized for CPU. In fact, most compiler profiles are not flat yet, and where your compilation takes a long time, it's because some optimization or algorithm has gone nuts and needs to be fixed. Until that kind of state of the world changes, measuring compile time is IMHO, a bit pointless, because they are only artificially CPU bound, and not in a way that is useful to benchmark (IE while i understand that people use these compilers every day, and care about performance, i'm just saying that, given a compiler benchmark, i could still probably make it take zero time with limited amounts of work)

rockdoe 10y ago
I think you're on the slippery slope here that lead to thinking Itanium was a good idea.
brandmeyer 10y ago
GCC is a useful benchmark not because it is unoptimized, but because it represents a real-world work profile that is otherwise poorly represented in SPEC. The working set sizes are large relative to the cache size, and its composed of a bunch of pointer-chasing branch-heavy logic as it winds through the various trees and intermediate languages. That profile is quite a bit closer to the profile of common business and server applications than nearly all of the other members in SPEC.
- DannyBee 10y ago
  "GCC is a useful benchmark not because it is unoptimized, but because it represents a real-world work profile that is otherwise poorly represented in SPEC." Sure, in that sense, but you also don't want benchmarks that are trivial to game by performing optimizations to them. :)
  - brandmeyer 10y ago
    For SPEC, you can't edit the source code of the unit under test. Only the compiler and host hardware count. Some components of SPEC have been "broken" by compilers in the past, by implementing some optimization or other that suddenly made that particular test code much faster. But you don't ever get to re-write the benchmark itself, you can't replace a 2^N path with an N^3 path or anything like that. So these kinds of breakages have almost always been restricted to benchmarks that are reducible to one or two relatively tight inner arithmetic-heavy loop(s). If someone managed to make an optimization that could suddenly make GCC 2x faster without humans editing GCC's source, that would be a major CS breakthrough :)

stefantalpalaru 10y ago

Phoronix does a "timed Linux kernel compilation" test where the 1800X is king: http://www.phoronix.com/scan.php?page=article&item=ryzen-1800x-linux&num=5 http://www.phoronix.com/scan.php?page=article&item=ryzen-180...