5 ms·

But less satisfying, it looks like clang optimizes out the entire zero-initialization logic, presumably because it assumes the expression ((size_t *)p)[-1] is u

• rightbyte 4y ago
  That is not undefined right? As long as p - 1 points into a valid object. Edit: Maybe Clang abuses the bitand with an int 7 to remove the code?
  • Someone 4y ago
    It is undefined behavior. The undefined behavior rules for pointers boil down to (https://stackoverflow.com/questions/56360316/c-standard-regarding-pointer-arithmetic-outside-arrays https://stackoverflow.com/questions/56360316/c-standard-rega...) “Pointer arithmetic is well defined only on array type and between array[0] and array[array_size+1]” And it’s even stricter: - you can create a pointer to array[array_size+1], but dereferencing it invokes undefined behavior. - changing a pointer that points into an object so that it points into another object invokes undefined behavior. Modern compilers track “pointer provenance” to sometimes speed up code (and occasionally perplex the programmer)
    • twoodfin 4y ago
      Those rules apply to access to the underlying array, they’re not a constraint on syntax for array expressions. malloc just returns a pointer, which could be in the middle of an array if malloc were an ordinary user-defined function. It’s only special knowledge of the behavior of standard malloc that allows the compiler to assume that pointer is to offset 0 of an allocated array.
      • wizeman 4y ago
        > malloc just returns a pointer, which could be in the middle of an array if malloc were an ordinary user-defined function. Yes, but even in that case, it would have to be an array of `size_t` objects for the pointer cast `(size_t *)` in the musl code not to trigger undefined behavior (especially if malloc would get inlined), right? Which means calloc() would trigger undefined behavior as soon as one of its callers used it for allocating a non-size_t array (especially if calloc() would get inlined with link-time optimization)?
        • Dylan16807 4y ago
          You just need certain parts of the returned allocation to be castable to size_t, and there's no way for the compiler to prove that wrong if it doesn't know the internals of the allocator. The underlying allocation could be as big as you want, and can be composed of whatever type you like.
          • wizeman 4y ago
            > You just need certain parts of the returned allocation to be castable to size_t, and there's no way for the compiler to prove that wrong if it doesn't know the internals of the allocator. The underlying allocation could be as big as you want, and can be composed of whatever type you like. Hmm, I don't think this is true. Even if you don't assume special semantics for malloc() and friends (by using the `-ffreestanding` and the `-fno-builtin` flags) and even if the compiler knows nothing about the malloc() implementation, I think if you're doing pointer arithmetic (with a pointer type other than `void *` or `[unsigned] char *`) then the compiler can assume that the pointer is pointing to either: 1) an actual array of objects of the underlying type (or a compatible one), or 2) a pointer to a single object, which is treated as an array with one element. So if you do `((size_t *) p)[-1]` within `calloc()`, then the compiler can assume that `*p`, if used, must either be a `size_t` (or a compatible integer) or `p` must be pointing to one past the end of a single-element array (which, since `*p` is a dereference, would trigger undefined behavior). Which means that if you call `calloc()` and it returns `p`, then you can only use it to dereference `size_t` objects or compatible ones (otherwise it would trigger undefined behavior). The effects of the undefined behavior would likely be observed if `calloc()` gets inlined into the caller (due to link-time optimization, for instance). Or am I wrong?
    • rightbyte 4y ago
      Funnily enough the SO answer has a off by one error. &array[array_size] is the last valid pointer (i.e. the pointer is valid, but can't be dereferenced).
    • MaxBarraclough 4y ago
      > changing a pointer that points into an object so that it points into another object invokes undefined behavior Do you have a source for this? Couldn't find anything on it with a quick web search.
      • Someone 4y ago
        https://en.cppreference.com/w/cpp/language/operator_arithmetic https://en.cppreference.com/w/cpp/language/operator_arithmet...: “If the pointer P points to the ith element of an array, then the expressions P + n, n + P, and P - n are pointers of the same type that point to the i+nth, i+nth, and i-nth element of the same array, respectively. The result of pointer addition may also be a one-past-the-end pointer (that is, pointer P such that the expression P - 1 points to the last element of the array). Any other situations (that is, attempts to generate a pointer that isn't pointing at an element of the same array or one past the end) invoke undefined behavior.”
        • MaxBarraclough 4y ago
          I think you may have misread my comment, that wasn't the part I was referring to.
• Sharlin 4y ago
  In the real world, musl would call its own `malloc` and can presumably trust that `((size_t *)p)[-1]` is valid and contains the block metadata (and the compiler can't assume it isn't valid), whereas the godbolt snippet calls libc's `malloc` which is a black box to outside callers. Or maybe LLVM can even deduce that the low bits are always zero on `libc`?
• leoetlino 4y ago
  Tried -fno-builtin-malloc on a hunch and it does seem to prevent the zero-initialisation logic from getting optimised away: https://godbolt.org/z/xWbEqcenn https://godbolt.org/z/xWbEqcenn musl probably uses -fno-builtin to disable any special handling of "builtin" library functions like malloc and memcpy?
• wizeman 4y ago
  > But less satisfying, it looks like clang optimizes out the entire zero-initialization logic, presumably because it assumes the expression ((size_t *)p)[-1] is undefined behavior. Yeah, I also noticed that is undefined behavior! > Makes me wonder what kind of compiler flags muslibc uses to prevent this. As far as I can see, the only related flags are `-ffreestanding`, `-fno-builtin` and `-fno-strict-aliasing` but I think none of these flags prevent the undefined behavior if 1) either the malloc() implementation gets inlined into calloc(), or 2) calloc() gets inlined into the caller, so that the compiler realizes that `p` is not actually an array of `size_t` or compatible integers. So perhaps musl also avoids undefined behavior by forcing `malloc()` and `calloc()` to not be inlined into the caller? Although I'm not entirely sure that actually completely prevents undefined behavior, I think it would just avoid it assuming how compilers usually/currently work? Edit: On further investigation, I think what I said above is wrong. I think `-ffreestanding` and `-fno-builtin` should be enough to prevent undefined behavior. I wasn't considering the case where `((size_t) *p)[-1]` points to a struct like this: struct alloc { size_t something; unsigned char buffer[]; // flexible array member } In this case, then I think the code is not triggering undefined behavior when used with those compiler flags (which prevent the compiler from using its knowledge about malloc's special semantics). Edit 2: On second thought, even then I think it might trigger undefined behavior because the C standard doesn't guarantee that there's no padding at the end of such a struct?
  • wizeman 4y ago
    So, I just checked and it seems that the parent pointed to a musl repository that contains old code. The current code doesn't contain that undefined behavior-triggering expression: https://git.musl-libc.org/cgit/musl/tree/src/malloc/calloc.c https://git.musl-libc.org/cgit/musl/tree/src/malloc/calloc.c
• tpolzer 4y ago
  Try with -ffreestanding. Otherwise clang is allowed to make assumptions about the behavior of "malloc". It then does the typical unsatisfying clang thing of unrolling and vectorizing a _lot_ with unclear benefits: https://godbolt.org/z/eG33YM7qs https://godbolt.org/z/eG33YM7qs