3 ms·

> You say no but then you list exceptions. You said "there's no way for the compiler to prove that wrong if it doesn't know the internals of the allocator" and

> You say no but then you list exceptions. You said "there's no way for the compiler to prove that wrong if it doesn't know the internals of the allocator" and I tried to reason how the compiler can prove that `*p` must be a `size_t` or compatible integer, even if it knows nothing about malloc() or its implementation. Which would trigger undefined behavior in the caller of calloc() if it weren't trying to allocate a size_t (or size_t array, or of compatible integers). Although normally this would only produce a visible effect if calloc() gets inlined into the caller. > Compatible integer types is easy. What do you mean? > And can't you also have any pointers you want into a char array as long as they are aligned correctly? I think so, but I think the problem is that when you do `(size_t *) p` (or pointer arithmetic with that) then the compiler can assume that p points to something that is not just a char array, which allows it to do more optimizations (due to the undefined behavior rules). > Also it should work to have a struct with a size_t followed by elements of a different type, shouldn't it? I think in general it wouldn't, because structure members are not guaranteed to be contiguous (i.e. they may have padding between them), so you may be triggering undefined behavior as well. Heck, they aren't even guaranteed to be laid out in memory in the same order as you define them (even though in practice that's what compilers do). However, if your struct has just a `size_t` field followed by a flexible array member then I think it would be OK! Edit: on second thought, even with a flexible array member I'm not sure it doesn't trigger undefined behavior because I think the C standard allows there to be padding at the end of such a struct. Which would be OK if the code was operating on a pointer to such a struct, but not to a pointer to size_t. So I think the code triggers undefined behavior even in this case.

Dylan16807 4y ago
> I think so, but I think the problem is that when you do `(size_t *) p` (or pointer arithmetic with that) then the compiler can assume that p points to something that is not just a char array, which allows it to do more optimizations (due to the undefined behavior rules). For the sake of clarity, I'm going to call `(size_t *) p` "q" for the rest of this post. Pointers usually have the type of what they point to. But pointers that are one past the end don't follow that rule. q-1 is a size_t, which means q is one past the end of an array containing one size_t. Therefore the existence of the q pointer does not tell the compiler what type actually exists at memory address p/q. Is it invalid to construct q from p? I wouldn't think so as long as they are in the same allocation, but maybe I'm forgetting a rule. > I think in general it wouldn't, because structure members are not guaranteed to be contiguous Surely there's a way to allocate two different types up against each other, though? Even if you don't make an actual struct, you can grab some char and say that X bytes are size_t and Y bytes are something else, can't you?
- wizeman 4y ago
  > Therefore the existence of the q pointer does not tell the compiler what type actually exists at memory address p/q. Is it invalid to construct q from p? I wouldn't think so as long as they are in the same allocation, but maybe I'm forgetting a rule. But then the question is, how would you construct an allocation where both q[-1] and *p are guaranteed to be valid objects of their respective types, using standard C? (i.e. without relying on implementation-specific behavior). > Surely there's a way to allocate two different types up against each other, though? If by "against each other" you mean without padding between them, then I think the C standard allows that but there's no reliable/portable/guaranteed way to do it (that I know of). By that I mean that specific implementations can guarantee that, either with struct packing or, in some cases, by carefully aligning them within a struct or a union. But this depends on the implementation/architecture, so it's not strictly portable/standard C, I think. > Even if you don't make an actual struct, you can grab some char and say that X bytes are size_t and Y bytes are something else, can't you? I'm not sure that is possible according to the standard, unless they already were objects of those types. See this[0]: > It is always presumed that a char* may refer to an alias of any object. It is therefore quite safe, if perhaps a bit unoptimal (for architecture with wide loads and stores) to cast any pointer of any type to a char* type. > The converse is not true. Casting a char* to a pointer of any type other than a char* and dereferencing it is usually in violation of the strict aliasing rule. > As noted by Pinskla it is not deferencing a char* per se that is specifically recognized as a potential alias of any object, but any address referring to a char object. This includes an array of char objects That said, I think for converting a bunch of chars to another type it's possible to get around that with memcpy() in some cases, but I'm not sure it would be helpful in this scenario. [0] https://cellperformance.beyond3d.com/articles/2006/06/understanding-strict-aliasing.html https://cellperformance.beyond3d.com/articles/2006/06/unders...
  - Dylan16807 4y ago
    How about this: We can make a struct and then have a function that bails out if for some reason the fields are not adjacent in the struct. Possibly even at compile time. In practice, this will work on basically any compiler. Once we know we have this struct, are there any remaining barriers to the cast and -1 access? >> Casting a char* to a pointer of any type other than a char* and dereferencing it is usually in violation of the strict aliasing rule. How does that logic work with malloc, which unlike calloc takes no item size and simply gives you a specific number of bytes?

Dylan16807 4y ago

> I think so, but I think the problem is that when you do `(size_t *) p` (or pointer arithmetic with that) then the compiler can assume that p points to something that is not just a char array, which allows it to do more optimizations (due to the undefined behavior rules). For the sake of clarity, I'm going to call `(size_t *) p` "q" for the rest of this post. Pointers usually have the type of what they point to. But pointers that are one past the end don't follow that rule. q-1 is a size_t, which means q is one past the end of an array containing one size_t. Therefore the existence of the q pointer does not tell the compiler what type actually exists at memory address p/q. Is it invalid to construct q from p? I wouldn't think so as long as they are in the same allocation, but maybe I'm forgetting a rule. > I think in general it wouldn't, because structure members are not guaranteed to be contiguous Surely there's a way to allocate two different types up against each other, though? Even if you don't make an actual struct, you can grab some char and say that X bytes are size_t and Y bytes are something else, can't you?

wizeman 4y ago
> Therefore the existence of the q pointer does not tell the compiler what type actually exists at memory address p/q. Is it invalid to construct q from p? I wouldn't think so as long as they are in the same allocation, but maybe I'm forgetting a rule. But then the question is, how would you construct an allocation where both q[-1] and *p are guaranteed to be valid objects of their respective types, using standard C? (i.e. without relying on implementation-specific behavior). > Surely there's a way to allocate two different types up against each other, though? If by "against each other" you mean without padding between them, then I think the C standard allows that but there's no reliable/portable/guaranteed way to do it (that I know of). By that I mean that specific implementations can guarantee that, either with struct packing or, in some cases, by carefully aligning them within a struct or a union. But this depends on the implementation/architecture, so it's not strictly portable/standard C, I think. > Even if you don't make an actual struct, you can grab some char and say that X bytes are size_t and Y bytes are something else, can't you? I'm not sure that is possible according to the standard, unless they already were objects of those types. See this[0]: > It is always presumed that a char* may refer to an alias of any object. It is therefore quite safe, if perhaps a bit unoptimal (for architecture with wide loads and stores) to cast any pointer of any type to a char* type. > The converse is not true. Casting a char* to a pointer of any type other than a char* and dereferencing it is usually in violation of the strict aliasing rule. > As noted by Pinskla it is not deferencing a char* per se that is specifically recognized as a potential alias of any object, but any address referring to a char object. This includes an array of char objects That said, I think for converting a bunch of chars to another type it's possible to get around that with memcpy() in some cases, but I'm not sure it would be helpful in this scenario. [0] https://cellperformance.beyond3d.com/articles/2006/06/understanding-strict-aliasing.html https://cellperformance.beyond3d.com/articles/2006/06/unders...
- Dylan16807 4y ago
  How about this: We can make a struct and then have a function that bails out if for some reason the fields are not adjacent in the struct. Possibly even at compile time. In practice, this will work on basically any compiler. Once we know we have this struct, are there any remaining barriers to the cast and -1 access? >> Casting a char* to a pointer of any type other than a char* and dereferencing it is usually in violation of the strict aliasing rule. How does that logic work with malloc, which unlike calloc takes no item size and simply gives you a specific number of bytes?