4 ms·

I am aware of the fact that unitilized padding can have nasty side effects: https://lwn.net/Articles/417989/ https://lwn.net/Articles/417989/ But yeah, I shoul

by unmole 8y ago

I am aware of the fact that unitilized padding can have nasty side effects: https://lwn.net/Articles/417989/ https://lwn.net/Articles/417989/ But yeah, I should have mentioned the caveats too. I will update the post once I'm back at my computer.

mehrdadn 8y ago
I'm confused what I'm supposed to look for in that article (do you mean you were the author?) but anyway -- I feel like calling this merely a "caveat" gives the wrong impression? It seems a bit like telling people to mix bleach with vinegar and then saying "Oops, sorry, I did know that it produces chlorine! I forgot to mention that caveat." The caveat isn't just a side note for the margins, it's a critical reason why people avoid this sort of thing.
- Dylan16807 8y ago
  Technically I don't think memset forces the compiler to clear the padding and keep it clear either though. The magic of the C abstract machine.
  - mehrdadn 8y ago
    Why? memset followed by memcpy must return the same byte everywhere. Otherwise what would memset even mean?
    - saagarjha 8y ago
      I feel like accessing the bytes that correspond to padding might be implementation-defined, but I couldn't find this in the standard.
      - mehrdadn 8y ago
        I don't think it is. It wouldn't make sense since memcpy is basically supposed to convert between arbitrary binary data.
        saagarjha 8y ago
        From a practical point of view, yes, but I would be very surprised if the C standard had any accommodation for memcpy being a way to leak the value of a structure's padding.
        jcr1488 8y ago
        You clearly haven't understood the thread.
        saagarjha 8y ago
        How so? Did I miss something important?
      - DougBTX 8y ago
        Parent comment was edited to include this quote: _6 When a value is stored in an object of structure or union type, including in a member object, the bytes of the object representation that correspond to any padding bytes take unspecified values. 51)_ So sure, a memcpy after a memset aught to preserve all the bytes, but if they are updated through a struct which contains padding then the values in the padding bytes will be left unspecified.
  - boulos 8y ago
    Do you mean in the sense that a compiler can say "I know what memset is supposed to do" then decides that even though sizeof returns a particular size, that it might ignore that and set fewer bytes when it does a substitution for the memcpy call?
    - Dylan16807 8y ago
      Yes. As long as the behavior observable by a conforming C program is the same, the compiler is allowed to change anything else. Even if memset in a vacuum is guaranteed, look at the quote by anyfoo. "When a value is stored in an object of structure or union type, including in a member object, the bytes of the object representation that correspond to any padding bytes take unspecified values." If the compiler knows you're going to write to the members right after the memset, it can put the padding back to its previous values. And by that I mean as far as your code can tell it put it back, but in actuality it never zeroed the padding to begin with. And since there are performance benefits to pretending, now you have a situation where a "helpful" compiler and a malicious compiler have the same effect: data can get leaked.
      - jcr1488 8y ago
        As soon as a pointer to said memory is passed to an extern function in another translation unit, the compiler can't prove anything about how it's used, which is the case in pretty much all of the examples mentioned in this thread. Also, type-punning is a thing. memset is byte-oriented/memory-oriented. Just because you're using it to zero a struct of a particular kind doesn't mean that's the only way the memory will accessed. Just because reading the padding of some struct is undefined behaviour doesn't mean accessing those bits by some other means is also undefined. A compiler usually can't eliminate a call to memset() in most practical cases of initialization (where memory leakage is also a concern) because they almost always pass a reference to a routine in another translation unit. Something like memzero_explicit() can be used anyway -- but you're massively overstating the relevance of compilers eliminating dead stores done via memset(). It's much more of an issue for post-destruction memory sanitizing (which is the primary use case for memzero_explicit) than it is for compiler f*ckery when memset() is used for initialization.
  - deleted 8y ago
    [deleted]
- ploxiln 8y ago
  Failing to clear padding in structs is _really_ not often a problem. It's pretty much only a problem when copying un-packed structs from kernel-space to user-space. So if you're not working on an OS kernel somewhere in the syscall path - really not a common activity even among C programmers - it's not a problem. If you are passing a struct from one part of your program to another part, or to a library you use which is not isolated or sandboxed in some way, it's not a problem. If you pass a struct to a _more_ privileged context like a syscall, it's not a problem. If you are passing structs over a network connection or writing to a file, you'll pack them and be aware of every byte anyway, in order to be compatible with another implementation reading them. If you _are_ passing a struct from a more privileged context to a less privileged context, then yeah you have to memset().
  - mehrdadn 8y ago
    I agree, but the frequency isn't the issue. You could say that's where my analogy breaks down, and if that's your point then maybe it does (although how often do people want to mix bleach with vinegar?), but that entirely misses the point. The point was that the failure mode, when it does come up, is a critical one, and if you're going to criticize a safe coding practice and tell people to switch to a potentially dangerous one, it behooves you to explain to them this very fact. It's just irresponsible to tell people to switch to a dangerous practice without making them aware of that fact and telling them how to deal with it.
  - kazinator 8y ago
    > If you pass a struct to a _more_ privileged context like a syscall, it's not a problem. Simply not true; that context could pass a copy of that structure to a less privileged context other than the original one, without doing anything about the padding. (Well, you could argue that it's the privileged kernel's fault: it should meticulously floss the structure between the members to clear the padding to zero while preserving the values.) Someone upthread mentioned "sending over a network": that's an example. Sending over a network (or local socket/pipe) begins with pushing it down to a more privileged context, which then assiduously places all the bytes into a buffer that is blasted out on the wire. In this case, the more privileged context has no idea what the structure even is; it's just a blob of bytes.
- bleair 8y ago
  If the author is only going to add "caveat", then that sounds like word games and being unwilling to admit that the blog post's advice is wrong (for the security risks and other reasons pointed out here in the comments)
deleted 8y ago
[deleted]

by unmole 8y ago

mehrdadn 8y ago

I'm confused what I'm supposed to look for in that article (do you mean you were the author?) but anyway -- I feel like calling this merely a "caveat" gives the wrong impression? It seems a bit like telling people to mix bleach with vinegar and then saying "Oops, sorry, I did know that it produces chlorine! I forgot to mention that caveat." The caveat isn't just a side note for the margins, it's a critical reason why people avoid this sort of thing.

Dylan16807 8y ago
Technically I don't think memset forces the compiler to clear the padding and keep it clear either though. The magic of the C abstract machine.
- mehrdadn 8y ago
  Why? memset followed by memcpy must return the same byte everywhere. Otherwise what would memset even mean?
  - saagarjha 8y ago
    I feel like accessing the bytes that correspond to padding might be implementation-defined, but I couldn't find this in the standard.
    - mehrdadn 8y ago
      I don't think it is. It wouldn't make sense since memcpy is basically supposed to convert between arbitrary binary data.
      - saagarjha 8y ago
        From a practical point of view, yes, but I would be very surprised if the C standard had any accommodation for memcpy being a way to leak the value of a structure's padding.
        jcr1488 8y ago
        You clearly haven't understood the thread.
        saagarjha 8y ago
        How so? Did I miss something important?
    - DougBTX 8y ago
      Parent comment was edited to include this quote: _6 When a value is stored in an object of structure or union type, including in a member object, the bytes of the object representation that correspond to any padding bytes take unspecified values. 51)_ So sure, a memcpy after a memset aught to preserve all the bytes, but if they are updated through a struct which contains padding then the values in the padding bytes will be left unspecified.
- boulos 8y ago
  Do you mean in the sense that a compiler can say "I know what memset is supposed to do" then decides that even though sizeof returns a particular size, that it might ignore that and set fewer bytes when it does a substitution for the memcpy call?
  - Dylan16807 8y ago
    Yes. As long as the behavior observable by a conforming C program is the same, the compiler is allowed to change anything else. Even if memset in a vacuum is guaranteed, look at the quote by anyfoo. "When a value is stored in an object of structure or union type, including in a member object, the bytes of the object representation that correspond to any padding bytes take unspecified values." If the compiler knows you're going to write to the members right after the memset, it can put the padding back to its previous values. And by that I mean as far as your code can tell it put it back, but in actuality it never zeroed the padding to begin with. And since there are performance benefits to pretending, now you have a situation where a "helpful" compiler and a malicious compiler have the same effect: data can get leaked.
    - jcr1488 8y ago
      As soon as a pointer to said memory is passed to an extern function in another translation unit, the compiler can't prove anything about how it's used, which is the case in pretty much all of the examples mentioned in this thread. Also, type-punning is a thing. memset is byte-oriented/memory-oriented. Just because you're using it to zero a struct of a particular kind doesn't mean that's the only way the memory will accessed. Just because reading the padding of some struct is undefined behaviour doesn't mean accessing those bits by some other means is also undefined. A compiler usually can't eliminate a call to memset() in most practical cases of initialization (where memory leakage is also a concern) because they almost always pass a reference to a routine in another translation unit. Something like memzero_explicit() can be used anyway -- but you're massively overstating the relevance of compilers eliminating dead stores done via memset(). It's much more of an issue for post-destruction memory sanitizing (which is the primary use case for memzero_explicit) than it is for compiler f*ckery when memset() is used for initialization.
- deleted 8y ago
  [deleted]
ploxiln 8y ago
Failing to clear padding in structs is _really_ not often a problem. It's pretty much only a problem when copying un-packed structs from kernel-space to user-space. So if you're not working on an OS kernel somewhere in the syscall path - really not a common activity even among C programmers - it's not a problem. If you are passing a struct from one part of your program to another part, or to a library you use which is not isolated or sandboxed in some way, it's not a problem. If you pass a struct to a _more_ privileged context like a syscall, it's not a problem. If you are passing structs over a network connection or writing to a file, you'll pack them and be aware of every byte anyway, in order to be compatible with another implementation reading them. If you _are_ passing a struct from a more privileged context to a less privileged context, then yeah you have to memset().
- mehrdadn 8y ago
  I agree, but the frequency isn't the issue. You could say that's where my analogy breaks down, and if that's your point then maybe it does (although how often do people want to mix bleach with vinegar?), but that entirely misses the point. The point was that the failure mode, when it does come up, is a critical one, and if you're going to criticize a safe coding practice and tell people to switch to a potentially dangerous one, it behooves you to explain to them this very fact. It's just irresponsible to tell people to switch to a dangerous practice without making them aware of that fact and telling them how to deal with it.
- kazinator 8y ago
  > If you pass a struct to a _more_ privileged context like a syscall, it's not a problem. Simply not true; that context could pass a copy of that structure to a less privileged context other than the original one, without doing anything about the padding. (Well, you could argue that it's the privileged kernel's fault: it should meticulously floss the structure between the members to clear the padding to zero while preserving the values.) Someone upthread mentioned "sending over a network": that's an example. Sending over a network (or local socket/pipe) begins with pushing it down to a more privileged context, which then assiduously places all the bytes into a buffer that is blasted out on the wire. In this case, the more privileged context has no idea what the structure even is; it's just a blob of bytes.
bleair 8y ago
If the author is only going to add "caveat", then that sounds like word games and being unwilling to admit that the blog post's advice is wrong (for the security risks and other reasons pointed out here in the comments)

deleted 8y ago

[deleted]