5 ms·

I'm no expert too, but please read this opinion about the problem [0]. I think he has a good point saying that undefined behavior != the compiler has the right

by GTP 10y ago

I'm no expert too, but please read this opinion about the problem [0]. I think he has a good point saying that undefined behavior != the compiler has the right to do wathewer it wants. [0] http://article.gmane.org/gmane.os.plan9.general/76989 http://article.gmane.org/gmane.os.plan9.general/76989

MaulingMonkey 10y ago
I'd agree with "undefined behavior != the compiler should do whatever it wants" - but I feel I must admit that the standard lets it, that many compiler authors apparently disagree with me, and that even those who do are sympathetic to my point of view or agree with me may have a point when they say that reason about programmer intention is a difficult bag of worms - and that preserving such, while maintaining "critical" optimizations, may not be as easy as it sounds. There's always -O0.
- mpweiher 10y ago
  There is a difference between "complies with the standard" and "is bug free". There were C compilers before there was a C standard. They had bugs.
  - MaulingMonkey 10y ago
    > There is a difference between "complies with the standard" and "is bug free". You're not wrong - but the dividing line between bug and feature request is subjective, and just how much (meaningful) difference between the two there is depends on the authors of the compiler. If you compile with -fno-strict-aliasing and the optimizer breaks your code based solely on strict aliasing violations anyways, by all means, report that as a bug. If you use a compiler that has no -fno-strict-aliasing equivalent, by all means, switch to a better compiler when they WONTFIX your feature request.
- GTP 10y ago
  Of course if the compiler has a certain behavior I have to deal with it, I just feel that certain optimizations based on undefined behavior are too aggressive, later this evening I will maybe write in more detail about this in anoter comment below.
  - to3m 10y ago
    My favourite UB-related links: "These people simply don't understand what C programmers want": https://groups.google.com/forum/#!msg/boring-crypto/48qa1kWignU/o8GGp2K1DAAJ https://groups.google.com/forum/#!msg/boring-crypto/48qa1kWi... "please don't do this, you're not producing value": http://blog.metaobject.com/2014/04/cc-osmartass.html http://blog.metaobject.com/2014/04/cc-osmartass.html "Everyone is fired": http://web.archive.org/web/20160309163927/http://robertoconcerto.blogspot.com/2010/10/strict-aliasing.html http://web.archive.org/web/20160309163927/http://robertoconc... (EDIT: this one just gets better every time I read it...) I also found a new one thanks to the comments here, which you can find elsewhere in the comments - but I'll add a link to it here anyway, for good measure: "No sane compiler writer would ever assume it allowed the compiler to 'do anything' with your code": http://article.gmane.org/gmane.os.plan9.general/76989 http://article.gmane.org/gmane.os.plan9.general/76989
    - kbenson 10y ago
      OMG, that first link is golden. It starts off by an email from DJB no less, which includes this snippet: I should note that this plan, throwing away gcc and clang in favor of a boring C compiler, isn't the only possible response to these types of security holes. Here are several other responses that I've seen: * Attack the messenger. "This code that you've written is undefined, so you're not allowed to comment on compiler behavior!" The most recent time I saw this, another language lawyer then jumped in to argue that the code in question _wasn't_ undefined---as if this side discussion had any relevance to the real issue. And to top it off, below Kurt Roeckx explains to someone what part of the problem is: The undefined behaviour of C is deliberate so that compilers can make optimazations. They assume you write code that only has defined meaning and generate code for that defined meaning. That means for instance that if you add 2 signed integers they're going to assume it doesn't overflow and then for instance make assumptions based on that on wether other code ever going to be executed or not. Niiice. This exact scenario is what happened and it's given as an example of problematic optimization of undefined behavior by compilers. If I felt it was pointless to really expect any change before, I definitely do now. I'm definitely favoriting your comment so I can find it easily later.
geocar 10y ago
I think it's a major problem with our industry that so many programmers are amused by being technically correct. Tradition notwithstanding, if a compiler deletes all my files because of the position of a comma in nested brackets, then I'm going to simply not use that compiler. I don't give a fuck if it's technically correct. "Oh it's not a bug". Good for you. I don't care if you want to call it a feature or a bug. It's wrong. Press the issue and I'll find someone smarter.
- jjnoakes 10y ago
  I don't know why you think this has anything to do with lack of intelligence.
- kutkloon7 10y ago
  It's true that I like to be technically correct. I don't consider this a problem itself. In fact, it can be quite important for programmers and mathematicians to be technically correct, as this bug in Cap'n Proto illustrates quite nicely. However, this is not the only reason I'm pointing out that this is not a compiler bug. The other reason is that there are certain implications: 1. The compiler developers are likely to declare that this is not their problem. 2. The problem may exist for other compilers as well: even if it complies to the standard it may do this. Even worse, it may only show up in a new version of the compiler, or in specific situations. I am not saying that this is a good situation and programmers should just be more careful. Everybody makes mistakes. The root of the problem is at the specification. Ideally, there would be no undefined behavior. From an optimizer's point of view it is very sensible to assume that the programmer will not invoke undefined behavior and use optimizations based on this principle. People surely love a compiler which produces fast code, so it may not be desirable to totally eradicate undefined behavior (and end this kind of optimizations). The most practical remedy I see is to add a debug flag which crashes or somehow indicates undefined behavior. Indeed, GCC has done this. So ultimately, we seem to agree that the compilers should change to improve this situation.
  - mannykannot 10y ago
    Agreed - I do not think the issue is with being technically correct, but in thinking that it always settles the matter and ends the discussion. It would seem particularly unfortunate if a compiler were to do static analysis for undefined behavior, but only use it in optimization. From a cursory search, I found this blog post [1] explaining why LLVM does not warn about such things (at least in 2011, when it was written - attitudes may have changed, and the Clang project now has the UndefinedBehaviorSanitizer [2]). One of the arguments is that it is difficult to explain what the problem is, but in my view, that is an argument for making doing so a priority. [1] http://blog.llvm.org/2011/05/what-every-c-programmer-should-know_21.html http://blog.llvm.org/2011/05/what-every-c-programmer-should-... [2] http://releases.llvm.org/3.8.0/tools/clang/docs/UndefinedBehaviorSanitizer.html http://releases.llvm.org/3.8.0/tools/clang/docs/UndefinedBeh...
kutkloon7 10y ago
undefined behavior = the compiler has the right to do wathever it wants This is exactly what the C standard says. And yes, that is problematic, especially since there is no way to detect undefined behaviour. I consider this to be the main problem. If C compilers would simply check for undefined behaviour in the debug build, there would be a lot less problems.
- vertex-four 10y ago
  The issue there is that you pretty much have to use undefined behaviour in C and certain low-level applications of C++. C is therefore a broken language IMHO, but... there we have it.
  - jjnoakes 10y ago
    I don't think one ever has to rely on undefined behavior. If you think you do, you can usually rewrite your code to avoid it, or rely on implementation defined behavior for specific compilers. And if that is not low level enough, you should be using assembly.
  - _pmf_ 10y ago
    That's wrong. All firmware I worked with used assembly for early initialization (startup.s) in those cases where the only alternative would be to rely on UB. I've worked with firmware that only worked with -O0 and failed with -O2, but these were always bad implementations, not due to some inherent requirement to rely on UB (most common example: failing to recognize that using volatile does not guarantee a memory barrier).
    - vertex-four 10y ago
      > I've worked with firmware that only worked with -O0 and failed with -O2 But as we can see in this vuln, there's functionality that works in the normal case but where important checks are elided, and compilers might decide not to do anything with a piece of undefined behaviour in one version but not in the next. Whether a piece of code works in -O2 doesn't prove it doesn't contain undefined behaviour.

by GTP 10y ago

MaulingMonkey 10y ago

I'd agree with "undefined behavior != the compiler should do whatever it wants" - but I feel I must admit that the standard lets it, that many compiler authors apparently disagree with me, and that even those who do are sympathetic to my point of view or agree with me may have a point when they say that reason about programmer intention is a difficult bag of worms - and that preserving such, while maintaining "critical" optimizations, may not be as easy as it sounds. There's always -O0.

mpweiher 10y ago
There is a difference between "complies with the standard" and "is bug free". There were C compilers before there was a C standard. They had bugs.
- MaulingMonkey 10y ago
  > There is a difference between "complies with the standard" and "is bug free". You're not wrong - but the dividing line between bug and feature request is subjective, and just how much (meaningful) difference between the two there is depends on the authors of the compiler. If you compile with -fno-strict-aliasing and the optimizer breaks your code based solely on strict aliasing violations anyways, by all means, report that as a bug. If you use a compiler that has no -fno-strict-aliasing equivalent, by all means, switch to a better compiler when they WONTFIX your feature request.
GTP 10y ago
Of course if the compiler has a certain behavior I have to deal with it, I just feel that certain optimizations based on undefined behavior are too aggressive, later this evening I will maybe write in more detail about this in anoter comment below.
- to3m 10y ago
  My favourite UB-related links: "These people simply don't understand what C programmers want": https://groups.google.com/forum/#!msg/boring-crypto/48qa1kWignU/o8GGp2K1DAAJ https://groups.google.com/forum/#!msg/boring-crypto/48qa1kWi... "please don't do this, you're not producing value": http://blog.metaobject.com/2014/04/cc-osmartass.html http://blog.metaobject.com/2014/04/cc-osmartass.html "Everyone is fired": http://web.archive.org/web/20160309163927/http://robertoconcerto.blogspot.com/2010/10/strict-aliasing.html http://web.archive.org/web/20160309163927/http://robertoconc... (EDIT: this one just gets better every time I read it...) I also found a new one thanks to the comments here, which you can find elsewhere in the comments - but I'll add a link to it here anyway, for good measure: "No sane compiler writer would ever assume it allowed the compiler to 'do anything' with your code": http://article.gmane.org/gmane.os.plan9.general/76989 http://article.gmane.org/gmane.os.plan9.general/76989
  - kbenson 10y ago
    OMG, that first link is golden. It starts off by an email from DJB no less, which includes this snippet: I should note that this plan, throwing away gcc and clang in favor of a boring C compiler, isn't the only possible response to these types of security holes. Here are several other responses that I've seen: * Attack the messenger. "This code that you've written is undefined, so you're not allowed to comment on compiler behavior!" The most recent time I saw this, another language lawyer then jumped in to argue that the code in question _wasn't_ undefined---as if this side discussion had any relevance to the real issue. And to top it off, below Kurt Roeckx explains to someone what part of the problem is: The undefined behaviour of C is deliberate so that compilers can make optimazations. They assume you write code that only has defined meaning and generate code for that defined meaning. That means for instance that if you add 2 signed integers they're going to assume it doesn't overflow and then for instance make assumptions based on that on wether other code ever going to be executed or not. Niiice. This exact scenario is what happened and it's given as an example of problematic optimization of undefined behavior by compilers. If I felt it was pointless to really expect any change before, I definitely do now. I'm definitely favoriting your comment so I can find it easily later.

geocar 10y ago

I think it's a major problem with our industry that so many programmers are amused by being technically correct. Tradition notwithstanding, if a compiler deletes all my files because of the position of a comma in nested brackets, then I'm going to simply not use that compiler. I don't give a fuck if it's technically correct. "Oh it's not a bug". Good for you. I don't care if you want to call it a feature or a bug. It's wrong. Press the issue and I'll find someone smarter.

jjnoakes 10y ago
I don't know why you think this has anything to do with lack of intelligence.
kutkloon7 10y ago
It's true that I like to be technically correct. I don't consider this a problem itself. In fact, it can be quite important for programmers and mathematicians to be technically correct, as this bug in Cap'n Proto illustrates quite nicely. However, this is not the only reason I'm pointing out that this is not a compiler bug. The other reason is that there are certain implications: 1. The compiler developers are likely to declare that this is not their problem. 2. The problem may exist for other compilers as well: even if it complies to the standard it may do this. Even worse, it may only show up in a new version of the compiler, or in specific situations. I am not saying that this is a good situation and programmers should just be more careful. Everybody makes mistakes. The root of the problem is at the specification. Ideally, there would be no undefined behavior. From an optimizer's point of view it is very sensible to assume that the programmer will not invoke undefined behavior and use optimizations based on this principle. People surely love a compiler which produces fast code, so it may not be desirable to totally eradicate undefined behavior (and end this kind of optimizations). The most practical remedy I see is to add a debug flag which crashes or somehow indicates undefined behavior. Indeed, GCC has done this. So ultimately, we seem to agree that the compilers should change to improve this situation.
- mannykannot 10y ago
  Agreed - I do not think the issue is with being technically correct, but in thinking that it always settles the matter and ends the discussion. It would seem particularly unfortunate if a compiler were to do static analysis for undefined behavior, but only use it in optimization. From a cursory search, I found this blog post [1] explaining why LLVM does not warn about such things (at least in 2011, when it was written - attitudes may have changed, and the Clang project now has the UndefinedBehaviorSanitizer [2]). One of the arguments is that it is difficult to explain what the problem is, but in my view, that is an argument for making doing so a priority. [1] http://blog.llvm.org/2011/05/what-every-c-programmer-should-know_21.html http://blog.llvm.org/2011/05/what-every-c-programmer-should-... [2] http://releases.llvm.org/3.8.0/tools/clang/docs/UndefinedBehaviorSanitizer.html http://releases.llvm.org/3.8.0/tools/clang/docs/UndefinedBeh...

kutkloon7 10y ago

undefined behavior = the compiler has the right to do wathever it wants This is exactly what the C standard says. And yes, that is problematic, especially since there is no way to detect undefined behaviour. I consider this to be the main problem. If C compilers would simply check for undefined behaviour in the debug build, there would be a lot less problems.

vertex-four 10y ago
The issue there is that you pretty much have to use undefined behaviour in C and certain low-level applications of C++. C is therefore a broken language IMHO, but... there we have it.
- jjnoakes 10y ago
  I don't think one ever has to rely on undefined behavior. If you think you do, you can usually rewrite your code to avoid it, or rely on implementation defined behavior for specific compilers. And if that is not low level enough, you should be using assembly.
- _pmf_ 10y ago
  That's wrong. All firmware I worked with used assembly for early initialization (startup.s) in those cases where the only alternative would be to rely on UB. I've worked with firmware that only worked with -O0 and failed with -O2, but these were always bad implementations, not due to some inherent requirement to rely on UB (most common example: failing to recognize that using volatile does not guarantee a memory barrier).
  - vertex-four 10y ago
    > I've worked with firmware that only worked with -O0 and failed with -O2 But as we can see in this vuln, there's functionality that works in the normal case but where important checks are elided, and compilers might decide not to do anything with a piece of undefined behaviour in one version but not in the next. Whether a piece of code works in -O2 doesn't prove it doesn't contain undefined behaviour.