3 ms·
The practical effect isn't what you think it is. Many compiler writers are employed by hardware companies, and these companies are employing them to sell their
by jcranmer 6y ago
The practical effect isn't what you think it is.
Many compiler writers are employed by hardware companies, and these companies are employing them to sell their hardware chips. For these companies, even a 1% boost in performance on $BENCHMARK can drive measurable impact on sales. Even if you're not worried about driving hardware sales, performance tends to be the major benchmark that gets mentioned in comparisons--note how the "Benchmarks Game" (what used to be called the Programming Language Shootout) compares languages solely on the basis of performance.
In other words, if you look at actual customer behavior, performance tends to be one of the most dominant factors in choosing one compiler over another (language compliance is probably more dominant, however). Surety of correctness is not a major factor at all.
Now, in order to actually report benchmarks correctly, you have to implement them correctly, so to some degree you have a basic guarantee that the compiler will usually work. But someone pointing out a soundness issue using a hypothetical example (as opposed to real code) is going to end up low priority because the evidence is that it doesn't matter in practice for the real, large applications that customers care about (as no one else has complained about it for the past several years).
In summary, practical performance will win out over theoretical correctness, but not practical correctness.
- igouy 6y ago> …"Benchmarks Game" … compares languages solely on the basis of performance. Not true. 1) source code size is compared https://benchmarksgame-team.pages.debian.net/benchmarksgame/performance/spectralnorm-gz.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/... https://benchmarksgame-team.pages.debian.net/benchmarksgame/how-programs-are-measured.html#source-code https://benchmarksgame-team.pages.debian.net/benchmarksgame/... 2) correctness is checked https://benchmarksgame-team.pages.debian.net/benchmarksgame/program/nbody-fsharpcore-1.html#log https://benchmarksgame-team.pages.debian.net/benchmarksgame/... 3) time "to complete and log all make actions" (build the program) is also logged
- jcranmer 6y ago> 1) source code size is compared This is new to me (the Benchmarks Game has been around for well over a decade). > 2) correctness is checked This is par for the course of benchmarks. A benchmark that is incorrectly compiled provides no useful information whatsoever about performance. It's not really what people mean when they complain about correctness issues in the compiler (which is generally referred to as soundness in literature). > 3) time "to complete and log all make actions" (build the program) is also logged Performance in a compiler is usually taken to be a combination of runtime, code size, and compile time, although all three components are not weighted equally.
- igouy 6y agoBack on May 23 2006, a sortable column of GZip source code size measurements: https://web.archive.org/web/20060523195137/http://shootout.alioth.debian.org/gp4/benchmark.php?test=spectralnorm&lang=all https://web.archive.org/web/20060523195137/http://shootout.a... https://web.archive.org/web/20060519213743/http://shootout.alioth.debian.org/gp4/faq.php#codelines https://web.archive.org/web/20060519213743/http://shootout.a... > Performance in a compiler is usually taken to be a combination of runtime, code size, and compile time The benchmarks game shows source code size, not compiled code size.