6 ms·

How interesting. GCC does indeed remove that branch. https://godbolt.org/z/aPcr1bfPe https://godbolt.org/z/aPcr1bfPe

mjg59 2y ago
Explanation for the above: passing NULL as the destination argument to memcpy() is undefined behaviour at present. gcc assumes that the fact that memcpy() is called therefore means that the destination argument can't be NULL, so "knows" that the dest == NULL check can never be true, and so removes the test and the do_thing1() branch entirely. Interestingly, replacing len in the memcpy() call results in gcc instead removing the memcpy() call and retaining the check - presumably a different optimisation routine decides that it's a no-op in that case. https://godbolt.org/z/cPdx6v13r https://godbolt.org/z/cPdx6v13r is, therefore, interesting - despite this only ever calling test() with a len of 0, the elision of the dest == NULL check is still there, but test() has been inlined without the memcpy (because len == 0) but with do_thing2() (because the behaviour is undefined and so it can assume dest isn't NULL even though there's a NULL literally right there!) Fucking compilers, man.
- nayuki 2y ago
  > Fucking compilers, man. They're just acting as agents that derive the logical consequences of the code. The fact that the given example code is "surprising" is analogous to this mathematical derivation: a = b a*a = b*a a*a - b*b = b*a - b*b (a - b)(a + b) = b(a - b) (a - b)(a + b)/(a - b) = b(a - b)/(a - b) ^ Divide by 0, undefined behavior! Everything below is not necessarily true. a + b = b b + b = b 2b = b 2 = 1 2 - 1 = 1 - 1 1 = 0 The source of truth about what is/isn't allowed is the C standard, not your personal simplified model of it that may contain dangerous misconceptions. The fact that your mental model doesn't match the document is an education problem, not a problem with the compiler.
  - marssaxman 2y ago
    > They're just acting as agents that derive the logical consequences of the code. In a particularly pedantic, uptight, and sometimes un-helpful way, yes. Compilers don't have to be designed this way; in fact it is a relatively recent development in the history of such tools.
  - saurik 2y ago
    > The fact that your mental model doesn't match the document is an education problem, not a problem with the compiler. Or it is a problem with the document, which is the entire reason we are having this discussion: N3322 argued the document should be fixed, and now it will be for C2y.
- jpollock 2y ago
  How does gcc infer anything about memcpy? Can't I replace the c-library memcpy with my own, so how does it know that dest == NULL can never be true?
  - 0xffff2 2y ago
    If I'm understanding the OP correctly, the C standard says so, i.e. the semantics of memcpy are defined by the standard and the standard says that it's UB to pass NULL.
    - tialaramex 2y ago
      Unlike all the more complicated languages the "freestanding" mode C doesn't even have a memcpy feature, so it may not define how one works - maybe you've decided to use the name "memcpy" for your function which generates a memorandum about large South American rodents, and "memo_capybara" was too much typing. In something like C++ or Rust, even their bare metal "What do you mean Operating System?" modes quietly require memcpy and so on because we're not savages, clearly somebody should provide a way to copy bytes of memory, Rust is so civilised that even on bare metal (in Rust's "core" library) you get a working sort_unstable() for your arbitrary slice types!
      - bonzini 2y ago
        The compiler is free to give a meaning to memcpy if run in the (default) hosted mode. There's -ffreestanding for freestanding environments.
        tialaramex 2y ago
        Right, though I guess I wasn't clear enough about that for the down voters, but whatever.
  - mjg59 2y ago
    The valid inputs to memcpy() are defined by the C specification, so the compiler is free to make assumptions about what valid inputs are even if the library implementation chooses to allow a broader range of inputs
- mpweiher 2y ago
  > that memcpy() is called therefore means that the destination argument can't be NULL The whole idea that undefined behavior cannot happen and you can therefore do optimization based on "knowing" it cannot happen is incredibly bonkers.
  - UncleMeat 2y ago
    Imagine this program. int foo() { int x = 1; havoc(); return x; } Can this function be compiled to store x in a register? Can it be compiled to remove x entirely and return the constant 1? That relies on "knowing that undefined behavior cannot happen." This program will behave differently if we store x on the stack and then return it after we call havoc() than if we call havoc() and then return the constant 1, if havoc() just writes to out of bounds memory addresses or whatever. In this case the undefined behavior just feels "more extreme" to most people, but it is remarkably hard for people to rigorously define the undefined behavior that should and should not be considered when making optimizations.
    - mpweiher 2y ago
      > That relies on "knowing that undefined behavior cannot happen." No it doesn't.
      - UncleMeat 2y ago
        Yes it does. The optimizing this to return the constant 1 is not producing an equivalent program unless we make assumptions about the behavioral bounds of havoc().
        mpweiher 2y ago
        That’s not at all the same thing.
        UncleMeat 2y ago
        What is the difference between "writing past the end of an array is UB" and "dereferencing a null pointer is UB" and "passing null as the destination argument to memcpy is UB"? The two programs I listed above are only observationally equivalent if writing past the end of valid allocations is UB. A core problem with this discussion in almost all circumstances is that people have a vibe for which of these things it feels okay for a compiler to make logical deductions from and which it feels not okay but if you actually sit down and try to formalize this in a way that would be meaningful to compiler vendors, you can't.
ndesaulniers 2y ago
> For example, GCC will happily remove the dest == NULL branch in the following code I think the blog should mention `-fno-delete-null-pointer-checks` https://gcc.gnu.org/onlinedocs/gcc/Optimize-Options.html#index-fdelete-null-pointer-checks https://gcc.gnu.org/onlinedocs/gcc/Optimize-Options.html#ind...
- AceJohnny2 2y ago
  > -fdelete-null-pointer-checks > [...] > This option is enabled by default on most targets. What a footgun. I understand that, in an effort to compete with other compilers for relevance, GCC pursued performance over safety. Has that era passed? Could GCC choose safer over fast? Alternatively, has someone compiled a list of flags one might want to enable in latest GCC to avoid such kinds of dangerous optimizations?
  - ryao 2y ago
    Usually, when one marks an argument as nonnull via a function attribute, one wants NULL checks to be removed.
    - ndesaulniers 2y ago
      There's two similar but distinct function attributes for nullability. One affects codegen, one affects diagnostics only.
      - ryao 2y ago
        Which are those? I only know about nonnull, nonnull_if_nonzero and returns_nonnull: https://gcc.gnu.org/onlinedocs/gcc/Common-Function-Attributes.html https://gcc.gnu.org/onlinedocs/gcc/Common-Function-Attribute...
    - AceJohnny2 2y ago
      Irrelevant, because delete-null-pointer-checks happens even in absence of nonnull function attribute, see GP's godbolt link, and the documentation that omits any reference to that function attribute. That's what makes it dangerous!

Explanation for the above: passing NULL as the destination argument to memcpy() is undefined behaviour at present. gcc assumes that the fact that memcpy() is called therefore means that the destination argument can't be NULL, so "knows" that the dest == NULL check can never be true, and so removes the test and the do_thing1() branch entirely. Interestingly, replacing len in the memcpy() call results in gcc instead removing the memcpy() call and retaining the check - presumably a different optimisation routine decides that it's a no-op in that case. https://godbolt.org/z/cPdx6v13r https://godbolt.org/z/cPdx6v13r is, therefore, interesting - despite this only ever calling test() with a len of 0, the elision of the dest == NULL check is still there, but test() has been inlined without the memcpy (because len == 0) but with do_thing2() (because the behaviour is undefined and so it can assume dest isn't NULL even though there's a NULL literally right there!) Fucking compilers, man.

nayuki 2y ago
> Fucking compilers, man. They're just acting as agents that derive the logical consequences of the code. The fact that the given example code is "surprising" is analogous to this mathematical derivation: a = b a*a = b*a a*a - b*b = b*a - b*b (a - b)(a + b) = b(a - b) (a - b)(a + b)/(a - b) = b(a - b)/(a - b) ^ Divide by 0, undefined behavior! Everything below is not necessarily true. a + b = b b + b = b 2b = b 2 = 1 2 - 1 = 1 - 1 1 = 0 The source of truth about what is/isn't allowed is the C standard, not your personal simplified model of it that may contain dangerous misconceptions. The fact that your mental model doesn't match the document is an education problem, not a problem with the compiler.
- marssaxman 2y ago
  > They're just acting as agents that derive the logical consequences of the code. In a particularly pedantic, uptight, and sometimes un-helpful way, yes. Compilers don't have to be designed this way; in fact it is a relatively recent development in the history of such tools.
- saurik 2y ago
  > The fact that your mental model doesn't match the document is an education problem, not a problem with the compiler. Or it is a problem with the document, which is the entire reason we are having this discussion: N3322 argued the document should be fixed, and now it will be for C2y.
jpollock 2y ago
How does gcc infer anything about memcpy? Can't I replace the c-library memcpy with my own, so how does it know that dest == NULL can never be true?
- 0xffff2 2y ago
  If I'm understanding the OP correctly, the C standard says so, i.e. the semantics of memcpy are defined by the standard and the standard says that it's UB to pass NULL.
  - tialaramex 2y ago
    Unlike all the more complicated languages the "freestanding" mode C doesn't even have a memcpy feature, so it may not define how one works - maybe you've decided to use the name "memcpy" for your function which generates a memorandum about large South American rodents, and "memo_capybara" was too much typing. In something like C++ or Rust, even their bare metal "What do you mean Operating System?" modes quietly require memcpy and so on because we're not savages, clearly somebody should provide a way to copy bytes of memory, Rust is so civilised that even on bare metal (in Rust's "core" library) you get a working sort_unstable() for your arbitrary slice types!
    - bonzini 2y ago
      The compiler is free to give a meaning to memcpy if run in the (default) hosted mode. There's -ffreestanding for freestanding environments.
      - tialaramex 2y ago
        Right, though I guess I wasn't clear enough about that for the down voters, but whatever.
- mjg59 2y ago
  The valid inputs to memcpy() are defined by the C specification, so the compiler is free to make assumptions about what valid inputs are even if the library implementation chooses to allow a broader range of inputs
mpweiher 2y ago
> that memcpy() is called therefore means that the destination argument can't be NULL The whole idea that undefined behavior cannot happen and you can therefore do optimization based on "knowing" it cannot happen is incredibly bonkers.
- UncleMeat 2y ago
  Imagine this program. int foo() { int x = 1; havoc(); return x; } Can this function be compiled to store x in a register? Can it be compiled to remove x entirely and return the constant 1? That relies on "knowing that undefined behavior cannot happen." This program will behave differently if we store x on the stack and then return it after we call havoc() than if we call havoc() and then return the constant 1, if havoc() just writes to out of bounds memory addresses or whatever. In this case the undefined behavior just feels "more extreme" to most people, but it is remarkably hard for people to rigorously define the undefined behavior that should and should not be considered when making optimizations.
  - mpweiher 2y ago
    > That relies on "knowing that undefined behavior cannot happen." No it doesn't.
    - UncleMeat 2y ago
      Yes it does. The optimizing this to return the constant 1 is not producing an equivalent program unless we make assumptions about the behavioral bounds of havoc().
      - mpweiher 2y ago
        That’s not at all the same thing.
        UncleMeat 2y ago
        What is the difference between "writing past the end of an array is UB" and "dereferencing a null pointer is UB" and "passing null as the destination argument to memcpy is UB"? The two programs I listed above are only observationally equivalent if writing past the end of valid allocations is UB. A core problem with this discussion in almost all circumstances is that people have a vibe for which of these things it feels okay for a compiler to make logical deductions from and which it feels not okay but if you actually sit down and try to formalize this in a way that would be meaningful to compiler vendors, you can't.

ndesaulniers 2y ago

> For example, GCC will happily remove the dest == NULL branch in the following code I think the blog should mention `-fno-delete-null-pointer-checks` https://gcc.gnu.org/onlinedocs/gcc/Optimize-Options.html#index-fdelete-null-pointer-checks https://gcc.gnu.org/onlinedocs/gcc/Optimize-Options.html#ind...

AceJohnny2 2y ago
> -fdelete-null-pointer-checks > [...] > This option is enabled by default on most targets. What a footgun. I understand that, in an effort to compete with other compilers for relevance, GCC pursued performance over safety. Has that era passed? Could GCC choose safer over fast? Alternatively, has someone compiled a list of flags one might want to enable in latest GCC to avoid such kinds of dangerous optimizations?
- ryao 2y ago
  Usually, when one marks an argument as nonnull via a function attribute, one wants NULL checks to be removed.
  - ndesaulniers 2y ago
    There's two similar but distinct function attributes for nullability. One affects codegen, one affects diagnostics only.
    - ryao 2y ago
      Which are those? I only know about nonnull, nonnull_if_nonzero and returns_nonnull: https://gcc.gnu.org/onlinedocs/gcc/Common-Function-Attributes.html https://gcc.gnu.org/onlinedocs/gcc/Common-Function-Attribute...
  - AceJohnny2 2y ago
    Irrelevant, because delete-null-pointer-checks happens even in absence of nonnull function attribute, see GP's godbolt link, and the documentation that omits any reference to that function attribute. That's what makes it dangerous!