4 ms·

Type punning is a mess in C because you have to jump through hoops to make it legal. The language subsequently failed to provide ergonomic solutions to the very

by Gankro 10y ago

Type punning is a mess in C because you have to jump through hoops to make it legal. The language subsequently failed to provide ergonomic solutions to the very real problems type punning solves in a systems language. Type punning is perfectly allowed in Rust, I'm not aware of any lints against it. Although you need to use an annotation to specify the struct layout algorithm to do it "correctly" for custom types. We use it in BTreeMap to implement a kind of inheritance between nodes. Internal nodes have the same layout as Leaf nodes, except internal nodes have an extra field for their array of edges (which you don't want to allocate for leaf nodes). So everything stores pointers to leaf nodes, and mostly manipulates all nodes as leaf nodes, but sometimes you "down cast" them to internal nodes to manipulate the edges. In this particular case we use the standard C++ pattern of making a LeafNode the first field of an InternalNode. https://doc.rust-lang.org/nightly/src/collections/up/src/libcollections/btree/node.rs.html#57-115 https://doc.rust-lang.org/nightly/src/collections/up/src/lib... The other common usage of punning in Rust is to gain access to some raw representation of a type. For instance, last time I worked on Rust, this was how fat pointers (&[T], &Trait) were constructed and decomposed at the lowest level. I can't speak to whether the usage of punning in this code is particularly good though.

wahern 10y ago
Type-punning is the not the same thing as deriving an object pointer through casting. Or relying on the rule concerning the equivalence of structures containing the same initial sequence of sub-members. Specifically, the following common macro in C is _not_ type-punning, at least not the kind I had in mind. #define container_of(ptr, type, member) \ (type *)((char *)(ptr) - offsetof(type, member)) Neither is the idiomatic BSD <sys/queue.h> library. It's all valid, well-defined code, as long as they're not being abused to hide undefined shenanigans. In C, as long as the last access to an object had the same type as the type you're accessing that object from (and provided it's the same object), that's perfectly well-defined. People think that this is an aliasing violation in C, but it's not. Aliasing issues only come into play when there are side-effects (including order of evaluation) that you implicitly depend on but that the compiler cannot see. That issue is too complex to bother discussing in fine detail here, but suffice it to say that Rust either has similar undefinedness issues, or it assumes any pointer however derived can alias even inside an unsafe block and therefore cannot perform the same optimizations that a C compiler can. I doubt the latter is the case given that rustc relies on the LLVM backend. Note that the general rule in C is that all pointers of the same type can alias, so if you derive two object pointers to the same type through explicit conversion (casting) or implicit conversion, and as long as they're actually referring to the same object, then as long as the last access is through the same type as the last store all is well-defined.[1] This is not type-punning. If this wasn't allowed by the language there wouldn't even be any use for casting at all. The cast is a way to stop the optimizer in its tracks and ensure that it doesn't fubar otherwise correct code. The aliasing issue typically comes into play when you access at least one object through a structure or union, and there's no union definition in scope that hints that the layout is such that the sub-members of the union or structure might alias. Though the dereferenced expressions might have the same type, the dereference isn't occurring through pointers with the proper compatible type. This is one of the few cases where the compiler isn't required to assume that accesses might alias. And this is why the C standard requires those types of evaluations to occur through pointers to a union. Type-punning, at least the kind I had in mind, is violating the core rule that access can only happen through an object with the same type as the last store. This is type-punning: unsigned long l = 1; unsigned *i; i = (unsigned *)&l; printf("i:%d\n", *i); The access through i has a different type than the store to l. An example that isn't type-punning per se, but raises the aliasing issue, struct foo { int i; } int add(struct foo *fp, int *ip) { int i; fp->i = 0; i = *ip; return fp->i + i; } int main(void) { struct foo f; return add(&f, &f.i); } In add(), the C compiler isn't required to assume that &fp->i might alias ip and so might reorder the statements. If the hidden dependency on ordering didn't exist, the code could otherwise be okay (that's why I don't call it type-punning). (Note: I'm having trouble using the asterisk without bolding everything. Please keep that in mind.) The above can be made correct simply by casting: int add(struct foo *fp, int *ip) { int i; *(int *)&fp->i = 0; i = *ip; return *(int *)&fp->i + i; } because now the initial store occurs through type pointer-to-integer, same as the type of ip. IOW, the compiler must assume that the store and loads might alias and cannot reorder things. As you can see, this is a much more contrived scenario. It's not as common as you'd think. It's most common when type-punning--storing through one type and accessing through another. Don't type-pun and you won't run into this issue very often, if ever. It's not even that common when using the typical C OOP-like inheritance tricks. It can happen in a silent and deadly way, but that requires some serious hackery. Don't pretend that C is Java and you're unlikely to write such code. One of the common places this occurs in practice is type-punning struct sockaddr, struct sockaddr_storage, etc. That's a very unique situation for many reasons. But as optimizations in compilers improve it is admittedly an increasing problem; it's a loaded gun, for sure. FWIW, C11 defines type-punning in a footnote 95 of section 6.5.2.3p3. If the member used to read the contents of a union object is not the same as the member last used to store a value in the object, the appropriate part of the object representation of the value is reinterpreted as an object representation in the new type as described in 6.2.6 (a process sometimes called ‘‘type punning’’). This might be a trap representation. This definition is even narrower than mine, but it still comports with the core rule about loads occurring through the same type as stores. [1] Storing a value through char is also okay as long as you ensure the representation is valid. Which is trivial when dealing with the standard fixed-width unsigned types. And access is always valid through char. That's why sometimes you'll see a seemingly superfluous cast through (char *). It's not necessarily type-punning; it might be used because a pointer-to-char can alias _anything_, and that can be useful as a barrier to prevent an optimizer from deciding two expressions might not alias.

by Gankro 10y ago

wahern 10y ago

Type-punning is the not the same thing as deriving an object pointer through casting. Or relying on the rule concerning the equivalence of structures containing the same initial sequence of sub-members. Specifically, the following common macro in C is _not_ type-punning, at least not the kind I had in mind. #define container_of(ptr, type, member) \ (type *)((char *)(ptr) - offsetof(type, member)) Neither is the idiomatic BSD <sys/queue.h> library. It's all valid, well-defined code, as long as they're not being abused to hide undefined shenanigans. In C, as long as the last access to an object had the same type as the type you're accessing that object from (and provided it's the same object), that's perfectly well-defined. People think that this is an aliasing violation in C, but it's not. Aliasing issues only come into play when there are side-effects (including order of evaluation) that you implicitly depend on but that the compiler cannot see. That issue is too complex to bother discussing in fine detail here, but suffice it to say that Rust either has similar undefinedness issues, or it assumes any pointer however derived can alias even inside an unsafe block and therefore cannot perform the same optimizations that a C compiler can. I doubt the latter is the case given that rustc relies on the LLVM backend. Note that the general rule in C is that all pointers of the same type can alias, so if you derive two object pointers to the same type through explicit conversion (casting) or implicit conversion, and as long as they're actually referring to the same object, then as long as the last access is through the same type as the last store all is well-defined.[1] This is not type-punning. If this wasn't allowed by the language there wouldn't even be any use for casting at all. The cast is a way to stop the optimizer in its tracks and ensure that it doesn't fubar otherwise correct code. The aliasing issue typically comes into play when you access at least one object through a structure or union, and there's no union definition in scope that hints that the layout is such that the sub-members of the union or structure might alias. Though the dereferenced expressions might have the same type, the dereference isn't occurring through pointers with the proper compatible type. This is one of the few cases where the compiler isn't required to assume that accesses might alias. And this is why the C standard requires those types of evaluations to occur through pointers to a union. Type-punning, at least the kind I had in mind, is violating the core rule that access can only happen through an object with the same type as the last store. This is type-punning: unsigned long l = 1; unsigned *i; i = (unsigned *)&l; printf("i:%d\n", *i); The access through i has a different type than the store to l. An example that isn't type-punning per se, but raises the aliasing issue, struct foo { int i; } int add(struct foo *fp, int *ip) { int i; fp->i = 0; i = *ip; return fp->i + i; } int main(void) { struct foo f; return add(&f, &f.i); } In add(), the C compiler isn't required to assume that &fp->i might alias ip and so might reorder the statements. If the hidden dependency on ordering didn't exist, the code could otherwise be okay (that's why I don't call it type-punning). (Note: I'm having trouble using the asterisk without bolding everything. Please keep that in mind.) The above can be made correct simply by casting: int add(struct foo *fp, int *ip) { int i; *(int *)&fp->i = 0; i = *ip; return *(int *)&fp->i + i; } because now the initial store occurs through type pointer-to-integer, same as the type of ip. IOW, the compiler must assume that the store and loads might alias and cannot reorder things. As you can see, this is a much more contrived scenario. It's not as common as you'd think. It's most common when type-punning--storing through one type and accessing through another. Don't type-pun and you won't run into this issue very often, if ever. It's not even that common when using the typical C OOP-like inheritance tricks. It can happen in a silent and deadly way, but that requires some serious hackery. Don't pretend that C is Java and you're unlikely to write such code. One of the common places this occurs in practice is type-punning struct sockaddr, struct sockaddr_storage, etc. That's a very unique situation for many reasons. But as optimizations in compilers improve it is admittedly an increasing problem; it's a loaded gun, for sure. FWIW, C11 defines type-punning in a footnote 95 of section 6.5.2.3p3. If the member used to read the contents of a union object is not the same as the member last used to store a value in the object, the appropriate part of the object representation of the value is reinterpreted as an object representation in the new type as described in 6.2.6 (a process sometimes called ‘‘type punning’’). This might be a trap representation. This definition is even narrower than mine, but it still comports with the core rule about loads occurring through the same type as stores. [1] Storing a value through char is also okay as long as you ensure the representation is valid. Which is trivial when dealing with the standard fixed-width unsigned types. And access is always valid through char. That's why sometimes you'll see a seemingly superfluous cast through (char *). It's not necessarily type-punning; it might be used because a pointer-to-char can alias _anything_, and that can be useful as a barrier to prevent an optimizer from deciding two expressions might not alias.