4 ms·

Given how aggressively compilers (including Clang) are exploiting undefined behavior these days, it is surprising to see Clang developers (I presume) on the bug

Given how aggressively compilers (including Clang) are exploiting undefined behavior these days, it is surprising to see Clang developers (I presume) on the bug being so blasé about introducing undefined behavior themselves.

electrograv 11y ago
> it is surprising to see Clang developers (I presume) on the bug being so blasé about introducing undefined behavior themselves. I have suddenly lost a great deal of respect and confidence for LLVM/Clang after reading this bug thread; I value software reliability above all else, and being able to write reliable software without the compiler negating that effort. It seems I can no longer be assured this is even a priority of my compiler any more, with Clang. All the more reason to switch to Rust I suppose. That the Clang people are trying to deflect this bug as "fine, no fix necessary" is kind of disgusting software engineering behavior. Compilers spontaneously introducing undefined behavior in another layer (to the point of causing codegen warnings from perfectly correct C code, is as clearly cut a BUG as any, independent of current API implementations outside of the ISO spec. If you want to make undefined API calls the norm, then make a new API where the behavior you want is defined. Invoking undefined behavior of a decoupled library that users can swap out is flat out indefensible. That they are reluctant to fix this makes me shy away from ever using Clang/LLVM, which is sad because before I had always held them with very high respect.
- chandlerc1024 11y ago
  Understand that some of the folks who don't think this is a bug own one of the libc implementations being used. So for their platform, it really is 100% correct and valid. The folks working on other platforms seem universally unhappy with that bug.
  - electrograv 11y ago
    If it's not 100% valid on 100% platforms and 100% of all future Libc revisions (which is impossible if it relies on undefined behavior and can link with different Libc versions), then it's not 100% correct and valid.
    - chandlerc1024 11y ago
      On at least one platform, all future libc revisions will abide by this requirement. And neither Clang nor LLVM are generating platform agnostic code. The code only has meaning for the specific platform being targeted.
  - xorblurb 11y ago
    It is not the particular situation people are upset about (well, at least me). It is the reasoning, especially in the context of "you shall not write undefined behavior according to the standard even if it should makes sens given the computer you are actually programming" that is usually sent on us regular programmer by compiler authors... :/ Yet they do exactly the same thing, and pretty much casually. And I actually don't really blame them for doing it. Everybody makes mistakes. But I blame them for the double standard, the risks, and the attitude. The problem in the first place: engineering is also about mitigating the fact that mistakes will happen, often mitigating even the mistake of others, and the modern approach in regard with undefined behavior does not help and is even ultra-hurtful for that. I don't see how stability will emerge with that kind of thought process.
    - chandlerc1024 11y ago
      You are tarring all compiler authors with a very broad brush here. Myself and other compiler authors I work with are much more specific in their requests. We do actually define some behavior that the standard doesn't require, and we do document this and tell our users its fine to rely on it. Unfortunately, we also have a lot of really clear experience with this stuff that lead us to two conclusions: 1) When we identify UB in code and go look at that code, for very large bodies of code what we find are overwhelmingly bugs. Bad, complete bugs. Defining the behavior wouldn't fix the bug, it was just one of the many symptoms. 2) We have real hardware differences for a lot of simple things that make them hard or impossible to give defined behavior for. As an example, shifting left by more bits than the size of the value gives different results on several different platforms. None of this means that we don't need to help programmers cope with this. We do indeed also make mistakes, and in fact it was these compiler authors that built a tool to help programmers find UB in their code: http://clang.llvm.org/docs/UndefinedBehaviorSanitizer.html http://clang.llvm.org/docs/UndefinedBehaviorSanitizer.html
      - wnoise 11y ago
        > We have real hardware differences for a lot of simple things that make them hard or impossible to give defined behavior for. Hard or impossible to give _performant_ defined behavior for. > As an example, shifting left by more bits than the size of the value gives different results on several different platforms. You could, for instance, always mod the shift by the word size, but possibly at a significant cost.
- archgoon 11y ago
  > All the more reason to switch to Rust I suppose. Rust is backed by llvm.
  - lomnakkus 11y ago
    Not particularly sure how that's particularly relevant here. This is an issue with Clang, AFAIUI.
    - archgoon 11y ago
      Well, electrograv was not distinguishing between the llvm project and the clang project. So it seemed reasonable to point out that rust was using llvm. > I have suddenly lost a great deal of respect and confidence for _LLVM_/Clang after reading this bug thread; However, do you believe that there is little overlap between llvm developers and clang developers?
chandlerc1024 11y ago
Hi, LLVM and Clang developer here. I don't see anyone being blasé about this. The issue comes down to something really simple: - The compiler can emit substantially simpler code if it has a guarantee from the platform's implementation of memcpy. - The compiler authors thought they had such a guarantee and so chose to leverage it. Now, maybe they don't have that guarantee on all platforms. If that's the case, its flat out and simply a bug in the compiler. And you'll find folks on that bug arguing that it doesn't actually hold, which is good and important. But it's actually quite reasonable to have a guarantee from a platform's libc that this will work and leverage that. The compiler isn't generating C code or code bound by any standard. This is about two parts of the implementation of C agreeing about what internal invariants will hold. The problem this bug highlights (and it is a real problem, and not one anyone takes lightly) is that we have insufficient agreement about these invariants, both for certain platforms (glibc) and even within LLVM. Fixing this remains absolutely critical, but may only involve documenting the agreement.
- haberman 11y ago
  Hi Chandler, I should have clarified that the comments I was referring to were on this bug: https://llvm.org/bugs/show_bug.cgi?id=11763 https://llvm.org/bugs/show_bug.cgi?id=11763 I was surprised to see: "I don't think it's really worth warning about, considering I don't know of any real-world implementation where it doesn't work." That sounds a lot like someone knowingly writing UB in a C program because no compiler they know of breaks the intended behavior. I doubt any memcpy() has ever guaranteed in any kind of official way that overlapping regions are ok, even if it has historically worked in practice. I mean I'm sure it'll get fixed now, I was just surprised to see this kind of reasoning coming from someone who I presumes works on the compiler. btw. I'm pretty sure I heard that glibc broke this behavior sometime recently. There's a new memcpy() symbol in glibc these days with a different symbol version that no longer provides correct behavior in the case of overlapping memory regions. This is all second-hand from some other people I was working with, but if I got their meaning correctly I think this issue will actually have relatively widespread impact.
  - chandlerc1024 11y ago

electrograv 11y ago

> it is surprising to see Clang developers (I presume) on the bug being so blasé about introducing undefined behavior themselves. I have suddenly lost a great deal of respect and confidence for LLVM/Clang after reading this bug thread; I value software reliability above all else, and being able to write reliable software without the compiler negating that effort. It seems I can no longer be assured this is even a priority of my compiler any more, with Clang. All the more reason to switch to Rust I suppose. That the Clang people are trying to deflect this bug as "fine, no fix necessary" is kind of disgusting software engineering behavior. Compilers spontaneously introducing undefined behavior in another layer (to the point of causing codegen warnings from perfectly correct C code, is as clearly cut a BUG as any, independent of current API implementations outside of the ISO spec. If you want to make undefined API calls the norm, then make a new API where the behavior you want is defined. Invoking undefined behavior of a decoupled library that users can swap out is flat out indefensible. That they are reluctant to fix this makes me shy away from ever using Clang/LLVM, which is sad because before I had always held them with very high respect.

chandlerc1024 11y ago
Understand that some of the folks who don't think this is a bug own one of the libc implementations being used. So for their platform, it really is 100% correct and valid. The folks working on other platforms seem universally unhappy with that bug.
- electrograv 11y ago
  If it's not 100% valid on 100% platforms and 100% of all future Libc revisions (which is impossible if it relies on undefined behavior and can link with different Libc versions), then it's not 100% correct and valid.
  - chandlerc1024 11y ago
    On at least one platform, all future libc revisions will abide by this requirement. And neither Clang nor LLVM are generating platform agnostic code. The code only has meaning for the specific platform being targeted.
- xorblurb 11y ago
  It is not the particular situation people are upset about (well, at least me). It is the reasoning, especially in the context of "you shall not write undefined behavior according to the standard even if it should makes sens given the computer you are actually programming" that is usually sent on us regular programmer by compiler authors... :/ Yet they do exactly the same thing, and pretty much casually. And I actually don't really blame them for doing it. Everybody makes mistakes. But I blame them for the double standard, the risks, and the attitude. The problem in the first place: engineering is also about mitigating the fact that mistakes will happen, often mitigating even the mistake of others, and the modern approach in regard with undefined behavior does not help and is even ultra-hurtful for that. I don't see how stability will emerge with that kind of thought process.
  - chandlerc1024 11y ago
    You are tarring all compiler authors with a very broad brush here. Myself and other compiler authors I work with are much more specific in their requests. We do actually define some behavior that the standard doesn't require, and we do document this and tell our users its fine to rely on it. Unfortunately, we also have a lot of really clear experience with this stuff that lead us to two conclusions: 1) When we identify UB in code and go look at that code, for very large bodies of code what we find are overwhelmingly bugs. Bad, complete bugs. Defining the behavior wouldn't fix the bug, it was just one of the many symptoms. 2) We have real hardware differences for a lot of simple things that make them hard or impossible to give defined behavior for. As an example, shifting left by more bits than the size of the value gives different results on several different platforms. None of this means that we don't need to help programmers cope with this. We do indeed also make mistakes, and in fact it was these compiler authors that built a tool to help programmers find UB in their code: http://clang.llvm.org/docs/UndefinedBehaviorSanitizer.html http://clang.llvm.org/docs/UndefinedBehaviorSanitizer.html
    - wnoise 11y ago
      > We have real hardware differences for a lot of simple things that make them hard or impossible to give defined behavior for. Hard or impossible to give _performant_ defined behavior for. > As an example, shifting left by more bits than the size of the value gives different results on several different platforms. You could, for instance, always mod the shift by the word size, but possibly at a significant cost.
archgoon 11y ago
> All the more reason to switch to Rust I suppose. Rust is backed by llvm.
- lomnakkus 11y ago
  Not particularly sure how that's particularly relevant here. This is an issue with Clang, AFAIUI.
  - archgoon 11y ago
    Well, electrograv was not distinguishing between the llvm project and the clang project. So it seemed reasonable to point out that rust was using llvm. > I have suddenly lost a great deal of respect and confidence for _LLVM_/Clang after reading this bug thread; However, do you believe that there is little overlap between llvm developers and clang developers?

chandlerc1024 11y ago

Hi, LLVM and Clang developer here. I don't see anyone being blasé about this. The issue comes down to something really simple: - The compiler can emit substantially simpler code if it has a guarantee from the platform's implementation of memcpy. - The compiler authors thought they had such a guarantee and so chose to leverage it. Now, maybe they don't have that guarantee on all platforms. If that's the case, its flat out and simply a bug in the compiler. And you'll find folks on that bug arguing that it doesn't actually hold, which is good and important. But it's actually quite reasonable to have a guarantee from a platform's libc that this will work and leverage that. The compiler isn't generating C code or code bound by any standard. This is about two parts of the implementation of C agreeing about what internal invariants will hold. The problem this bug highlights (and it is a real problem, and not one anyone takes lightly) is that we have insufficient agreement about these invariants, both for certain platforms (glibc) and even within LLVM. Fixing this remains absolutely critical, but may only involve documenting the agreement.

haberman 11y ago
Hi Chandler, I should have clarified that the comments I was referring to were on this bug: https://llvm.org/bugs/show_bug.cgi?id=11763 https://llvm.org/bugs/show_bug.cgi?id=11763 I was surprised to see: "I don't think it's really worth warning about, considering I don't know of any real-world implementation where it doesn't work." That sounds a lot like someone knowingly writing UB in a C program because no compiler they know of breaks the intended behavior. I doubt any memcpy() has ever guaranteed in any kind of official way that overlapping regions are ok, even if it has historically worked in practice. I mean I'm sure it'll get fixed now, I was just surprised to see this kind of reasoning coming from someone who I presumes works on the compiler. btw. I'm pretty sure I heard that glibc broke this behavior sometime recently. There's a new memcpy() symbol in glibc these days with a different symbol version that no longer provides correct behavior in the case of overlapping memory regions. This is all second-hand from some other people I was working with, but if I got their meaning correctly I think this issue will actually have relatively widespread impact.
- chandlerc1024 11y ago