8 ms·

Can someone ELI5 what the issue is with type punning and union aliasing?

by drdrey 8y ago

ufo 8y ago
Suppose you want to look at the bits of a floating point number, as though it were an integer. The type punning way to do that would be to access a double in memory through an integer pointer: double d = 3.14; int64_t *p = (int64_t *) &d; // !!! int64_t n = *p; The union aliasing way would be to makeset the double field in a union and read the integer field back: typedef union { double d; int64_t n; } int_or_double; int_or_double u; u.d = 3.14 int64_t n = u.n According to the standard, the union aliasing approach is allowed, but the type punning one is undefined behavior. --------- The standard team decided to forbid type punning to allow compilers to implement more optimizations. They still allow union aliasing because they felt there should still be a way to do low-level type casts if for those that absolutely want to.
- blorsh 8y ago
  No, type punning via unions is a gcc extension. The standard only allows reading a value as the type it was written with or as char, and an access as char is only good for copying. If a char access used different bit order, that would be OK according to the standard because you couldn't tell unless you violated the standard. It seems every compiler will tolerate a memcpy for type punning, even though this isn't required.
  - Athas 8y ago
    Does this mean that memcpy() is really more of a language construct than a library function, since it cannot be implemented correctly in fully defined standard C?
    - adrianN 8y ago
      Yes. Check here for a discussion: https://blog.regehr.org/archives/959 https://blog.regehr.org/archives/959
    - gpderetta 8y ago
      Everything defined in the standard is in principle a language construct: whether it is builtin in the compiler or an actual function is an implementation detail. For many compilers, memcpy is both: they have builtin handling for it but fall back to the library implementation when there is no optimization opportunities. Having said that, it is possible to implement memcopy in standard C: char pointers have an explicit special exemption to the aliasing rule so you are allowed to inspect objects as array of chars.
  - ensiferum 8y ago
    memcpy is legal and fine. To borrow the example from above: double d = 3.14; uint64_t i; memcpy(&i, &d, sizeof(d)); is perfectly kosher.
    - minipci1321 8y ago
      > is perfectly kosher Aren't exact-bitwidth integer types optional even with C99 (let alone preceding versions of standard)?
      - boomlinde 8y ago
        They are optional, but only in the sense that if you can't exactly represent e.g. uint64_t, there can't be a uint64_t type. So the example above is kosher in the sense that it's standards compliant, even if it isn't perfectly portable (for reasons unrelated to memset type punning; that you're using uint64_t in the first place). If an implementation can't represent uint64_t exactly, it can still have a uint_least64_t type of whatever width works.
        deleted 8y ago
        [deleted]
        minipci1321 8y ago
        I certainly agree with what you said as you said it. However, as usual, it is the implementation that kills it. a) I believe sizeof (double) * 8 == 64 is not enforced by the standard either. One could argue that it is hard to find anything else these days, but (for a contrived example) I believe nothing prevents you from making your own port of GCC for some purpose, and make 'double' be 'float'. (From my memories, it is even far enough from rocket science.) b) It already happened to me personally, to work with code where people have implemented their own uintXX types (because of the lack of the standard ones), and then over time these became narrower/vider. So if we would like to have it portable, I think, nothing beats: union { double d; char cc [sizeof (double)]; }; and there we are, back to what is in the standard.
  - ufo 8y ago
    C99 and C11 allow type punning via unions (although apparently that only gets mentioned in a footnote) https://stackoverflow.com/questions/11639947/ https://stackoverflow.com/questions/11639947/
- acqq 8y ago
  AFAIK even using union for the above purpose can be seen as "undefined behavior" when reading standard, and that's why the standard is especially wrong, as it makes some typical desired (and needed) outcomes impossible to implement, and at the moment clang still has problems even with the union variant whereas gcc allows you to use union by passing the compiler switch which effectively has the semantics "we promise we won't listen to the standard in this case." That's why Linus writes: "So the commit message that talks about how horrible union aliasing is is pushing a story that is simply wrong." The commit message with which he disagreed obviously explicitly wrote something like (paraphrasing) "using union is undefined behavior by standard, so we are changing the code to not use it" etc. Linus also wrote: "I'm not talking about the changes themselves - I can live with them. But the _rationale_ is pure and utter garbage, and dangerously so." The rationale was (note, I don't have exact quote, I'm paraphrasing, if somebody has the original please give a link) that using unions for that is bad as per standard. Which is what the standard says but which is for practical purposes simply wrong. The standard is wrong an should not be followed, that was main Linus' argument.
  - boomlinde 8y ago
    What part of the standard should I read to come to the conclusion that type punning with unions is undefined behavior? C99 and C11 both explicitly mention that it's not, but maybe you're talking C89/C90? IMO there are 4 highly relevant paragraphs in C99: 6.2.6.1: When a value is stored in an object of structure or union type, including in a member object, the bytes of the object representation that correspond to any padding bytes take unspecified values. 6.2.6.1: When a value is stored in a member of an object of union type, the bytes of the object representation that do not correspond to that member but do correspond to other members take unspecified values 6.5.2.3 DR283) If the member used to access the contents of a union object is not the same as the member last used to store a value in the object, the appropriate part of the object representation of the value is reinterpreted as an object representation in the new type as described in 6.2.6 (a process sometimes called "type punning"). 6.7.2.1: A pointer to a union object, suitably converted, points to each of its members (or if a member is a bit-field, then to the unit in which it resides), and vice versa. None of this implies that unions can't be used for type punning. What it does imply is that reading from a larger member of the union than you last wrote to will leave you with an unspecified value.
pubby 8y ago
Several compiler optimizations center around proving which variables don't change. Allowing type punning makes it harder for the compiler to do this.

Can someone ELI5 what the issue is with type punning and union aliasing?

by drdrey 8y ago

Can someone ELI5 what the issue is with type punning and union aliasing?

ufo 8y ago

Suppose you want to look at the bits of a floating point number, as though it were an integer. The type punning way to do that would be to access a double in memory through an integer pointer: double d = 3.14; int64_t *p = (int64_t *) &d; // !!! int64_t n = *p; The union aliasing way would be to makeset the double field in a union and read the integer field back: typedef union { double d; int64_t n; } int_or_double; int_or_double u; u.d = 3.14 int64_t n = u.n According to the standard, the union aliasing approach is allowed, but the type punning one is undefined behavior. --------- The standard team decided to forbid type punning to allow compilers to implement more optimizations. They still allow union aliasing because they felt there should still be a way to do low-level type casts if for those that absolutely want to.

blorsh 8y ago
No, type punning via unions is a gcc extension. The standard only allows reading a value as the type it was written with or as char, and an access as char is only good for copying. If a char access used different bit order, that would be OK according to the standard because you couldn't tell unless you violated the standard. It seems every compiler will tolerate a memcpy for type punning, even though this isn't required.
- Athas 8y ago
  Does this mean that memcpy() is really more of a language construct than a library function, since it cannot be implemented correctly in fully defined standard C?
  - adrianN 8y ago
    Yes. Check here for a discussion: https://blog.regehr.org/archives/959 https://blog.regehr.org/archives/959
  - gpderetta 8y ago
    Everything defined in the standard is in principle a language construct: whether it is builtin in the compiler or an actual function is an implementation detail. For many compilers, memcpy is both: they have builtin handling for it but fall back to the library implementation when there is no optimization opportunities. Having said that, it is possible to implement memcopy in standard C: char pointers have an explicit special exemption to the aliasing rule so you are allowed to inspect objects as array of chars.
- ensiferum 8y ago
  memcpy is legal and fine. To borrow the example from above: double d = 3.14; uint64_t i; memcpy(&i, &d, sizeof(d)); is perfectly kosher.
  - minipci1321 8y ago
    > is perfectly kosher Aren't exact-bitwidth integer types optional even with C99 (let alone preceding versions of standard)?
    - boomlinde 8y ago
      They are optional, but only in the sense that if you can't exactly represent e.g. uint64_t, there can't be a uint64_t type. So the example above is kosher in the sense that it's standards compliant, even if it isn't perfectly portable (for reasons unrelated to memset type punning; that you're using uint64_t in the first place). If an implementation can't represent uint64_t exactly, it can still have a uint_least64_t type of whatever width works.
      - deleted 8y ago
        [deleted]
      - minipci1321 8y ago
        I certainly agree with what you said as you said it. However, as usual, it is the implementation that kills it. a) I believe sizeof (double) * 8 == 64 is not enforced by the standard either. One could argue that it is hard to find anything else these days, but (for a contrived example) I believe nothing prevents you from making your own port of GCC for some purpose, and make 'double' be 'float'. (From my memories, it is even far enough from rocket science.) b) It already happened to me personally, to work with code where people have implemented their own uintXX types (because of the lack of the standard ones), and then over time these became narrower/vider. So if we would like to have it portable, I think, nothing beats: union { double d; char cc [sizeof (double)]; }; and there we are, back to what is in the standard.
- ufo 8y ago
  C99 and C11 allow type punning via unions (although apparently that only gets mentioned in a footnote) https://stackoverflow.com/questions/11639947/ https://stackoverflow.com/questions/11639947/
acqq 8y ago
AFAIK even using union for the above purpose can be seen as "undefined behavior" when reading standard, and that's why the standard is especially wrong, as it makes some typical desired (and needed) outcomes impossible to implement, and at the moment clang still has problems even with the union variant whereas gcc allows you to use union by passing the compiler switch which effectively has the semantics "we promise we won't listen to the standard in this case." That's why Linus writes: "So the commit message that talks about how horrible union aliasing is is pushing a story that is simply wrong." The commit message with which he disagreed obviously explicitly wrote something like (paraphrasing) "using union is undefined behavior by standard, so we are changing the code to not use it" etc. Linus also wrote: "I'm not talking about the changes themselves - I can live with them. But the _rationale_ is pure and utter garbage, and dangerously so." The rationale was (note, I don't have exact quote, I'm paraphrasing, if somebody has the original please give a link) that using unions for that is bad as per standard. Which is what the standard says but which is for practical purposes simply wrong. The standard is wrong an should not be followed, that was main Linus' argument.
- boomlinde 8y ago
  What part of the standard should I read to come to the conclusion that type punning with unions is undefined behavior? C99 and C11 both explicitly mention that it's not, but maybe you're talking C89/C90? IMO there are 4 highly relevant paragraphs in C99: 6.2.6.1: When a value is stored in an object of structure or union type, including in a member object, the bytes of the object representation that correspond to any padding bytes take unspecified values. 6.2.6.1: When a value is stored in a member of an object of union type, the bytes of the object representation that do not correspond to that member but do correspond to other members take unspecified values 6.5.2.3 DR283) If the member used to access the contents of a union object is not the same as the member last used to store a value in the object, the appropriate part of the object representation of the value is reinterpreted as an object representation in the new type as described in 6.2.6 (a process sometimes called "type punning"). 6.7.2.1: A pointer to a union object, suitably converted, points to each of its members (or if a member is a bit-field, then to the unit in which it resides), and vice versa. None of this implies that unions can't be used for type punning. What it does imply is that reading from a larger member of the union than you last wrote to will leave you with an unspecified value.

pubby 8y ago

Several compiler optimizations center around proving which variables don't change. Allowing type punning makes it harder for the compiler to do this.