4 ms·

You're not going to discover compiler bugs if you stick to language features and compilers that have been thoroughly trodden, and don't push the limits in any s

You're not going to discover compiler bugs if you stick to language features and compilers that have been thoroughly trodden, and don't push the limits in any special way (massive functions etc..).

cmrx64 11y ago
That's simply not true. There are some bugs in GCC with flow-sensitive constant evaluation that cause statically-impossible "undefined behavior" to be seen and aggressively optimized against, causing a miscompilation. I've encountered clang and gcc internal errors on rather kosher modestly sized C++ projects as well.
- Ono-Sendai 11y ago
  The GCC devs do seem to be pushing hard on undefined behaviour optimisation, sometimes a little too hard, resulting in the introduction of bugs. I have come across a lot of transient miscompilation in visual studio, I guess, usually when doing incremental rebuilds. I was thinking more of persistent miscompilations as things you are unlikely to find on the 'well trodden path'
  - to3m 11y ago
    I found Visual Studio a lot more reliable once I took to disabling Minimal Rebuild. (Don't ask me why I thought to try this... it's possible I read about it somewhere.) People always seemed to leave this switched on, because it sounds like you want it - but you don't. It supplants the standard file time-based dependency checking with some additional content-based stuff (see https://msdn.microsoft.com/en-us/library/kfz8ad09.aspx https://msdn.microsoft.com/en-us/library/kfz8ad09.aspx) that goes wrong every now and again.
    - Ono-Sendai 11y ago
      I always have /GM off because it interferes with multi-threaded building. However I still get miscompilations / bad builds occasionally, which are fixed with a full rebuild.
comex 11y ago
You're less likely to discover compiler bugs, but you can still "beat the odds" if your codebase is big enough. Many of the bugs mentioned in the post were triggered by code not doing anything especially unusual.
derf_ 11y ago
For an example of what Greg Maxwell might consider "adequate testing", see https://www.ietf.org/proceedings/82/slides/codec-4.pdf https://www.ietf.org/proceedings/82/slides/codec-4.pdf (he is the one responsible for all of the testing described in that presentation). Slide 20 lists no less than 8 toolchain bugs found with a codebase of less than 50k lines of C code (no C++), using only thoroughly trodden language features, without "massive functions" or anything else. Admittedly 4 of those bugs were in pre-release versions of gcc, but the rest weren't.
- Ono-Sendai 11y ago
  I've heard ICC is pretty buggy. Tinycc seems a bit obscure. pre-release versions of GCC as you mentioned are not well-trodden.
  - nkurz 11y ago
    I've heard ICC is pretty buggy. Perhaps historically ICC was buggy, but I don't think it's true anymore. I often test highly optimized C code with GCC, Clang, and ICC, and anecdotally I'd say that the likelihood of hitting a compiler bug when compiling for a current Intel processor is about the same for each. For me, crashing bugs and true miscompilation are rare with all three, but come up occasionally. Performance differences are usually within +/- 20% on microbenchmarks, with each having about equal chances of being the fastest or slowest.
TillE 11y ago
I had a massive memory leak in normal usage of std::stringstream in Visual C++ 2008, which eventually got fixed in an update. Wasted a lot of my time tracking down that leak, though.
- Ono-Sendai 11y ago
  stringstream is maybe one of those terrible APIs that bugs remain in because barely anyone uses them :)
  - PeCaN 11y ago
    I don't think I'd consider std::stringstream to be particularly obscure.
    - scott_s 11y ago
      Particularly since it's the easiest way to implement a generic to_string function (prior to C++11).
  - GFK_of_xmaspast 11y ago
    I use stringstream all the time to prepare complicated output for a logger.
MaulingMonkey 11y ago
Compilers are a moving target - and you'd be surprised what you can find. Here's one I recently ran into (with a different repro:) https://connect.microsoft.com/VisualStudio/feedback/details/758711/incorrect-expansion-of-line-with-enabled-edit-and-continue-line-var https://connect.microsoft.com/VisualStudio/feedback/details/... I didn't file a bug as it'd already been reported - although in my case, the trigger was using a logging macro which used __LINE__ from a lambda IIRC. Nothing terribly spectacular. The last bug I remember reporting was default constructor parameters not instantiating templates correctly. Something like void f(int i = some_template<int>::func()); would fail to generate, and thus link against, some_template<int>::func. f might've needed to be a constructor. Sadly the connect bug is 404 and web.archive.org didn't capture the page. They said they would fix it in VS2008 SP2, but I ended up checking and finding they'd fixed it in SP1. Even ignoring ICEs, I'd say I probably hit at least one confirmed compiler bug a year minimum when doing regular C++ development.
mikeash 11y ago
My favorite compiler bug was clang occasionally failing to set the al register to a sensible value before making a vararg call. It was especially fun because clang's prologue for vararg functions was robust against bad values, so nothing happened for clang->clang calls. But call into a vararg function compiled by gcc and all hell broke loose when gcc used the value of al as part of a computed jump that assumed it was in the range of 0-8. The circumstances needed to trigger this bug weren't special, I never nailed it down entirely but it was just something like performing an integer division at just the wrong spot. Vararg function calls are extremely well trodden, and clang at the time was pretty mature. Yet there was a bug, all the same.
sbierwagen 11y ago
Generally, if you run into a bug while programming, it is probably a mistake you, yourself, made. This is not always true, compiler bugs do exist, and can be found: http://blog.regehr.org/archives/category/compilers http://blog.regehr.org/archives/category/compilers Consider that entire industries (aerospace, etc) routinely ship code with compiler optimization disabled, to avoid getting bit by compilers trying to do clever things.
- Ono-Sendai 11y ago
  I would generally agree, unless you are coding in OpenCL :)
cpeterso 11y ago
Compiler changes in code generation and optimization can affect even well-trodden code. The article mentions that the following line of code would, in some cases, only zero the first four bytes of the array when compiled with VS2015: char key[5] = { 0 }; and the "fix": // Work around VC++ 2015 Update 1 code-gen bug: // https://connect.microsoft.com/VisualStudio/feedback/details/2291638 key[4] = 0; https://chromium.googlesource.com/chromium/third_party/ffmpeg.git/+/2f698edc4eb0819432c332508c15c0aec374bea2%5E%21/ https://chromium.googlesource.com/chromium/third_party/ffmpe...
- burfog 11y ago
  It was a Chromium bug, and it still isn't fixed because Microsoft wrongly accepted it as a compiler bug. The Chromium code contained this gem involving a union type called av_alias32 containing an int: (((av_alias32*)(key))->u32 = (argc)); The first problem is that an int may have an alignment constraint. Since writing an int to an allocation that is made based on char could violate this constraint, the code is invalid. It doesn't matter that x86 won't actually enforce this. The code is still invalid. Since that line of code is invalid, all code paths that will reach it are also invalid, as are all code paths from it. This makes the entire test program invalid; it is legit for the compiler to emit a program that immediately crashes or insults your mom. The second problem is when the char array is checked to see if it contains the 42. An access as char is always permitted, so the code narrowly misses having an aliasing violation, but remember that the type AS ACTUALLY WRITTEN is not char. The only thing that can be legitimately done with the value is to write it elsewhere, for example when implementing a function like memcpy. An int need not be stored in little-endian form, big-endian form, two's complement, or anything else sane... and the language standard allows for this by prohibiting what this code is doing. As with the other bug: all code paths which involve the standards violation are invalid, making the whole test program invalid. The compiler is permitted to emit code that crashes or insults your mom.
  - brucedawson 11y ago
    I'm sure you are correct about the aliasing being undefined behavior. However, Microsoft's attitude towards certain types of undefined is quite different from the gcc/clang attitude and VC++ (more or less) promise not to break your program if you break the aliasing rules. gcc/clang are within their rights to break programs that break the aliasing rules, but VC++ is not required to do so and they choose not to. There have been some excellent articles talking about how the current state of undefined behavior as understood by gcc/clang is crazy. One that I like is this one: http://blog.regehr.org/archives/761 http://blog.regehr.org/archives/761 Obviously some types of undefined behavior cannot be 'normalized', such as use-after-free, but treating the aliasing in that bug as undefined behavior is a committee decision that many developers think is ill advised. Another recent article that I can't find pokes some holes in the idea that the latitude given to compilers by undefined behavior improves performance. For your second example of undefined behavior I'm afraid I don't understand. Are you saying that even if the line of code that aliases the array is removed that "if (key[0] != 42)" is still undefined behavior? If so please explain. You say "the type AS ACTUALLY WRITTEN is not char" but I'm not sure what type you are referring to, and if you mean '42' that is true but not relevant because in a comparison between key[0] and '42' "key[0]" is converted to int by the standard integral conversions and then a particularly boring comparison is done. If that code is undefined then the standards committee is definitely crazy and no code is conforming.
nickpsecurity 11y ago
The other side of that coin is that one of my high-assurance system recommendations was specifically to use the most common constructs and libraries in most common configurations. Albeit with safety or interface checks added where necessary. Even without provable correctness, it dramatically reduces occurrence of all kinds of issues.
mappu 11y ago
I independently found an internal compiler error in GCC 4.5.2 while using inline assembly. By the time i managed to create a small test case, it was fixed in a new upstream GCC.
cyphar 11y ago
My team has found several bugs in gcc-go (we maintain Docker for architectures that don't have a port of the gc compiler). Now, you might argue that Docker is full of weird stuff (you'd be correct), but the bugs we found were not related to any of the really weird stuff in Docker (it was mainly with cgo IIRC).

You're not going to discover compiler bugs if you stick to language features and compilers that have been thoroughly trodden, and don't push the limits in any special way (massive functions etc..).

cmrx64 11y ago

That's simply not true. There are some bugs in GCC with flow-sensitive constant evaluation that cause statically-impossible "undefined behavior" to be seen and aggressively optimized against, causing a miscompilation. I've encountered clang and gcc internal errors on rather kosher modestly sized C++ projects as well.

Ono-Sendai 11y ago
The GCC devs do seem to be pushing hard on undefined behaviour optimisation, sometimes a little too hard, resulting in the introduction of bugs. I have come across a lot of transient miscompilation in visual studio, I guess, usually when doing incremental rebuilds. I was thinking more of persistent miscompilations as things you are unlikely to find on the 'well trodden path'
- to3m 11y ago
  I found Visual Studio a lot more reliable once I took to disabling Minimal Rebuild. (Don't ask me why I thought to try this... it's possible I read about it somewhere.) People always seemed to leave this switched on, because it sounds like you want it - but you don't. It supplants the standard file time-based dependency checking with some additional content-based stuff (see https://msdn.microsoft.com/en-us/library/kfz8ad09.aspx https://msdn.microsoft.com/en-us/library/kfz8ad09.aspx) that goes wrong every now and again.
  - Ono-Sendai 11y ago
    I always have /GM off because it interferes with multi-threaded building. However I still get miscompilations / bad builds occasionally, which are fixed with a full rebuild.

comex 11y ago

You're less likely to discover compiler bugs, but you can still "beat the odds" if your codebase is big enough. Many of the bugs mentioned in the post were triggered by code not doing anything especially unusual.

derf_ 11y ago

For an example of what Greg Maxwell might consider "adequate testing", see https://www.ietf.org/proceedings/82/slides/codec-4.pdf https://www.ietf.org/proceedings/82/slides/codec-4.pdf (he is the one responsible for all of the testing described in that presentation). Slide 20 lists no less than 8 toolchain bugs found with a codebase of less than 50k lines of C code (no C++), using only thoroughly trodden language features, without "massive functions" or anything else. Admittedly 4 of those bugs were in pre-release versions of gcc, but the rest weren't.

Ono-Sendai 11y ago
I've heard ICC is pretty buggy. Tinycc seems a bit obscure. pre-release versions of GCC as you mentioned are not well-trodden.
- nkurz 11y ago
  I've heard ICC is pretty buggy. Perhaps historically ICC was buggy, but I don't think it's true anymore. I often test highly optimized C code with GCC, Clang, and ICC, and anecdotally I'd say that the likelihood of hitting a compiler bug when compiling for a current Intel processor is about the same for each. For me, crashing bugs and true miscompilation are rare with all three, but come up occasionally. Performance differences are usually within +/- 20% on microbenchmarks, with each having about equal chances of being the fastest or slowest.

TillE 11y ago

I had a massive memory leak in normal usage of std::stringstream in Visual C++ 2008, which eventually got fixed in an update. Wasted a lot of my time tracking down that leak, though.

Ono-Sendai 11y ago
stringstream is maybe one of those terrible APIs that bugs remain in because barely anyone uses them :)
- PeCaN 11y ago
  I don't think I'd consider std::stringstream to be particularly obscure.
  - scott_s 11y ago
    Particularly since it's the easiest way to implement a generic to_string function (prior to C++11).
- GFK_of_xmaspast 11y ago
  I use stringstream all the time to prepare complicated output for a logger.

MaulingMonkey 11y ago

Compilers are a moving target - and you'd be surprised what you can find. Here's one I recently ran into (with a different repro:) https://connect.microsoft.com/VisualStudio/feedback/details/758711/incorrect-expansion-of-line-with-enabled-edit-and-continue-line-var https://connect.microsoft.com/VisualStudio/feedback/details/... I didn't file a bug as it'd already been reported - although in my case, the trigger was using a logging macro which used __LINE__ from a lambda IIRC. Nothing terribly spectacular. The last bug I remember reporting was default constructor parameters not instantiating templates correctly. Something like void f(int i = some_template<int>::func()); would fail to generate, and thus link against, some_template<int>::func. f might've needed to be a constructor. Sadly the connect bug is 404 and web.archive.org didn't capture the page. They said they would fix it in VS2008 SP2, but I ended up checking and finding they'd fixed it in SP1. Even ignoring ICEs, I'd say I probably hit at least one confirmed compiler bug a year minimum when doing regular C++ development.

mikeash 11y ago

My favorite compiler bug was clang occasionally failing to set the al register to a sensible value before making a vararg call. It was especially fun because clang's prologue for vararg functions was robust against bad values, so nothing happened for clang->clang calls. But call into a vararg function compiled by gcc and all hell broke loose when gcc used the value of al as part of a computed jump that assumed it was in the range of 0-8. The circumstances needed to trigger this bug weren't special, I never nailed it down entirely but it was just something like performing an integer division at just the wrong spot. Vararg function calls are extremely well trodden, and clang at the time was pretty mature. Yet there was a bug, all the same.

sbierwagen 11y ago

Generally, if you run into a bug while programming, it is probably a mistake you, yourself, made. This is not always true, compiler bugs do exist, and can be found: http://blog.regehr.org/archives/category/compilers http://blog.regehr.org/archives/category/compilers Consider that entire industries (aerospace, etc) routinely ship code with compiler optimization disabled, to avoid getting bit by compilers trying to do clever things.

Ono-Sendai 11y ago
I would generally agree, unless you are coding in OpenCL :)

cpeterso 11y ago

Compiler changes in code generation and optimization can affect even well-trodden code. The article mentions that the following line of code would, in some cases, only zero the first four bytes of the array when compiled with VS2015: char key[5] = { 0 }; and the "fix": // Work around VC++ 2015 Update 1 code-gen bug: // https://connect.microsoft.com/VisualStudio/feedback/details/2291638 key[4] = 0; https://chromium.googlesource.com/chromium/third_party/ffmpeg.git/+/2f698edc4eb0819432c332508c15c0aec374bea2%5E%21/ https://chromium.googlesource.com/chromium/third_party/ffmpe...

burfog 11y ago
It was a Chromium bug, and it still isn't fixed because Microsoft wrongly accepted it as a compiler bug. The Chromium code contained this gem involving a union type called av_alias32 containing an int: (((av_alias32*)(key))->u32 = (argc)); The first problem is that an int may have an alignment constraint. Since writing an int to an allocation that is made based on char could violate this constraint, the code is invalid. It doesn't matter that x86 won't actually enforce this. The code is still invalid. Since that line of code is invalid, all code paths that will reach it are also invalid, as are all code paths from it. This makes the entire test program invalid; it is legit for the compiler to emit a program that immediately crashes or insults your mom. The second problem is when the char array is checked to see if it contains the 42. An access as char is always permitted, so the code narrowly misses having an aliasing violation, but remember that the type AS ACTUALLY WRITTEN is not char. The only thing that can be legitimately done with the value is to write it elsewhere, for example when implementing a function like memcpy. An int need not be stored in little-endian form, big-endian form, two's complement, or anything else sane... and the language standard allows for this by prohibiting what this code is doing. As with the other bug: all code paths which involve the standards violation are invalid, making the whole test program invalid. The compiler is permitted to emit code that crashes or insults your mom.
- brucedawson 11y ago
  I'm sure you are correct about the aliasing being undefined behavior. However, Microsoft's attitude towards certain types of undefined is quite different from the gcc/clang attitude and VC++ (more or less) promise not to break your program if you break the aliasing rules. gcc/clang are within their rights to break programs that break the aliasing rules, but VC++ is not required to do so and they choose not to. There have been some excellent articles talking about how the current state of undefined behavior as understood by gcc/clang is crazy. One that I like is this one: http://blog.regehr.org/archives/761 http://blog.regehr.org/archives/761 Obviously some types of undefined behavior cannot be 'normalized', such as use-after-free, but treating the aliasing in that bug as undefined behavior is a committee decision that many developers think is ill advised. Another recent article that I can't find pokes some holes in the idea that the latitude given to compilers by undefined behavior improves performance. For your second example of undefined behavior I'm afraid I don't understand. Are you saying that even if the line of code that aliases the array is removed that "if (key[0] != 42)" is still undefined behavior? If so please explain. You say "the type AS ACTUALLY WRITTEN is not char" but I'm not sure what type you are referring to, and if you mean '42' that is true but not relevant because in a comparison between key[0] and '42' "key[0]" is converted to int by the standard integral conversions and then a particularly boring comparison is done. If that code is undefined then the standards committee is definitely crazy and no code is conforming.

nickpsecurity 11y ago

The other side of that coin is that one of my high-assurance system recommendations was specifically to use the most common constructs and libraries in most common configurations. Albeit with safety or interface checks added where necessary. Even without provable correctness, it dramatically reduces occurrence of all kinds of issues.

mappu 11y ago

I independently found an internal compiler error in GCC 4.5.2 while using inline assembly. By the time i managed to create a small test case, it was fixed in a new upstream GCC.

cyphar 11y ago

My team has found several bugs in gcc-go (we maintain Docker for architectures that don't have a port of the gc compiler). Now, you might argue that Docker is full of weird stuff (you'd be correct), but the bugs we found were not related to any of the really weird stuff in Docker (it was mainly with cgo IIRC).