2 ms·

Hi Chandler, I should have clarified that the comments I was referring to were on this bug: https://llvm.org/bugs/show_bug.cgi?id=11763 https://llvm.org/bugs/s

Hi Chandler, I should have clarified that the comments I was referring to were on this bug: https://llvm.org/bugs/show_bug.cgi?id=11763 https://llvm.org/bugs/show_bug.cgi?id=11763 I was surprised to see: "I don't think it's really worth warning about, considering I don't know of any real-world implementation where it doesn't work." That sounds a lot like someone knowingly writing UB in a C program because no compiler they know of breaks the intended behavior. I doubt any memcpy() has ever guaranteed in any kind of official way that overlapping regions are ok, even if it has historically worked in practice. I mean I'm sure it'll get fixed now, I was just surprised to see this kind of reasoning coming from someone who I presumes works on the compiler. btw. I'm pretty sure I heard that glibc broke this behavior sometime recently. There's a new memcpy() symbol in glibc these days with a different symbol version that no longer provides correct behavior in the case of overlapping memory regions. This is all second-hand from some other people I was working with, but if I got their meaning correctly I think this issue will actually have relatively widespread impact.

chandlerc1024 11y ago
That comment was actually a considered comment: when we look at what warnings we should produce, we consider cases where everyone has written code a particular way and the fact that it isn't defined is actually a language bug. We (the Clang community members, and I suspect the GCC community members as well) have also worked to change language standards to provide guarantees relied on consistently and where the lack of guarantee has no reasonable basis. Also that comment is very old, and predates us learning about all of why this is probably something actually worth warning on. That discussion ended up in a different forum, and we do actually warn on stuff like self assignment. Understand that it is often precisely the people working on a compiler that do deal with the cases of when we discover widespread UB in real code and whether or not the code is incorrect and must be changed or the standard is incorrect and must be fixed. But all of that is about a warning, and not about the actual question of how the compiler and the system library interact. Note that both Clang and GCC have at least some variants of this bug, and so I don't actually expect glibc to break this in the foreseeable future. But we're still actively working to defend against this on platforms for which we don't have a guarantee.
- haberman 11y ago
  I can't see an argument here for why the standard should be changed. memcpy() does one thing and memmove() does another. Changing the standard so they are the same just because memcpy() gets misused in real-world programs doesn't make sense to me. If people misuse memcpy(), then have a compiler flag akin to -fstrict-aliasing where memcpy() gets rewritten to memmove() unless people compile with -fstrict-memcpy. That seems a separate issue though from what calls the compiler should emit. I am not following why LLVM would emit memcpy() when the semantics it actually requires from libc are memmove(). Am I missing something?
  - Joky 11y ago
    It is not about making memcpy behave the same as memmove: in this case the difference would still be between exact identity (src == dest) and partial overlap.
    - haberman 11y ago
      I see, thanks for clarifying. I agree there could be reason to consider a change in that case.
- colin_mccabe 11y ago
  LLVM is a very impressive compiler. I'm really impressed with the warning messages it's able to produce (especially compared with what was available even 2 or 3 years ago). What is more frustrating about modern compilers (and this is not limited to LLVM) is the desire to squeeze out a tiny bit more performance by exploiting undefined behavior. I'm thinking of things like eliminating calls to memset that "don't seem to do anything user-visible," eliminating code based on signed overflow being undefined, and so on. Most of the the undefined behavior that exists in C is there because different machines used to do different things. For example, there used to be non-two's-complement machines. It wasn't put there because C was somehow designed to be hyper-optimized. People who relied on the undefined behavior could legitimately make the same argument that you are now making-- that it worked fine on their machine, which they knew very well. I am curious why nobody has brought up the idea of making memmove as efficiently handled as memset, rather than using undefined behavior here. UB feels like the wrong way to go.
anon1385 11y ago
>btw. I'm pretty sure I heard that glibc broke this behavior sometime recently. You are perhaps thinking of this: https://sourceware.org/bugzilla/show_bug.cgi?id=12518 https://sourceware.org/bugzilla/show_bug.cgi?id=12518 I'm not sure if that change broke glibc memcpy for identical arguments or just ones that were different but overlapped. This quote suggests that Drepper wouldn't have had any problem breaking things that passed identical arguments to memcpy: >The existence of code written by people who should never have been allowed to touch a keyboard cannot be allowed to prevent a correct implementation.

chandlerc1024 11y ago

That comment was actually a considered comment: when we look at what warnings we should produce, we consider cases where everyone has written code a particular way and the fact that it isn't defined is actually a language bug. We (the Clang community members, and I suspect the GCC community members as well) have also worked to change language standards to provide guarantees relied on consistently and where the lack of guarantee has no reasonable basis. Also that comment is very old, and predates us learning about all of why this is probably something actually worth warning on. That discussion ended up in a different forum, and we do actually warn on stuff like self assignment. Understand that it is often precisely the people working on a compiler that do deal with the cases of when we discover widespread UB in real code and whether or not the code is incorrect and must be changed or the standard is incorrect and must be fixed. But all of that is about a warning, and not about the actual question of how the compiler and the system library interact. Note that both Clang and GCC have at least some variants of this bug, and so I don't actually expect glibc to break this in the foreseeable future. But we're still actively working to defend against this on platforms for which we don't have a guarantee.

haberman 11y ago
I can't see an argument here for why the standard should be changed. memcpy() does one thing and memmove() does another. Changing the standard so they are the same just because memcpy() gets misused in real-world programs doesn't make sense to me. If people misuse memcpy(), then have a compiler flag akin to -fstrict-aliasing where memcpy() gets rewritten to memmove() unless people compile with -fstrict-memcpy. That seems a separate issue though from what calls the compiler should emit. I am not following why LLVM would emit memcpy() when the semantics it actually requires from libc are memmove(). Am I missing something?
- Joky 11y ago
  It is not about making memcpy behave the same as memmove: in this case the difference would still be between exact identity (src == dest) and partial overlap.
  - haberman 11y ago
    I see, thanks for clarifying. I agree there could be reason to consider a change in that case.
colin_mccabe 11y ago
LLVM is a very impressive compiler. I'm really impressed with the warning messages it's able to produce (especially compared with what was available even 2 or 3 years ago). What is more frustrating about modern compilers (and this is not limited to LLVM) is the desire to squeeze out a tiny bit more performance by exploiting undefined behavior. I'm thinking of things like eliminating calls to memset that "don't seem to do anything user-visible," eliminating code based on signed overflow being undefined, and so on. Most of the the undefined behavior that exists in C is there because different machines used to do different things. For example, there used to be non-two's-complement machines. It wasn't put there because C was somehow designed to be hyper-optimized. People who relied on the undefined behavior could legitimately make the same argument that you are now making-- that it worked fine on their machine, which they knew very well. I am curious why nobody has brought up the idea of making memmove as efficiently handled as memset, rather than using undefined behavior here. UB feels like the wrong way to go.

anon1385 11y ago

>btw. I'm pretty sure I heard that glibc broke this behavior sometime recently. You are perhaps thinking of this: https://sourceware.org/bugzilla/show_bug.cgi?id=12518 https://sourceware.org/bugzilla/show_bug.cgi?id=12518 I'm not sure if that change broke glibc memcpy for identical arguments or just ones that were different but overlapped. This quote suggests that Drepper wouldn't have had any problem breaking things that passed identical arguments to memcpy: >The existence of code written by people who should never have been allowed to touch a keyboard cannot be allowed to prevent a correct implementation.