4 ms·

Surely you only need memset_s if you're doing this for security reasons, as if memset is optimized away it's because the compiler can prove the observable seman

Surely you only need memset_s if you're doing this for security reasons, as if memset is optimized away it's because the compiler can prove the observable semantics are the same.

olliej 7y ago
Nope, memset will often be “optimized” such that padding is not zeroed. It’s not because the compiler sees that padding isn’t observed, it’s because reading from padding is undefined. Because reading from padding is undefined the write (also undefined) by definition has no observable side effects. Memset_s is the only way to guarantee that padding is actually zeroed, well memset or bzero_s obviously. It’s critically important for people to understand that if the C spec says doing X is undefined, then current compiler writers interpret this as allowing them to do anything they want, even when it clearly introduces security vulnerabilities.
- eridius 7y ago
  Right, but zeroing padding only matters for security reasons. If your memset isn't being done for security reasons, then memset() is better than memset_s().
  - olliej 7y ago
    If you are processing data from the network, any uninitialized read gets attacked. My experience is padding (both inside a struct, and outside - array alignment for instance) gets attacked. The only safe option is to initialize everything. I say this having spent more than a decade working on software that is always being attacked.
    - Matheus28 7y ago
      You shouldn't be sending structs over the network, since their specific layout depends on the compiler (and version). At least make them packed if you're gonna do undefined behavior... But you should just do it the right way and have a wrapper that deals with endianness differences and such: struct S my_struct; // ... push_int64(&output_stream, &my_struct.value1); push_string(&output_stream, my_struct.value2); push_int32(&output_stream, &my_struct.value3); And on the receiving side: struct S my_struct; next_int64(&input_stream, &my_struct.value1); next_string(&input_stream, my_struct.value2, sizeof(my_struct.value2)); next_int32(&input_stream, &my_struct.value3);
      - olliej 7y ago
        The problem is not sending structs across the network. The problem is processing any data from the network. You have to assume that data is malicious, and will attack any weakness. I’m not talking about serialisation at all - I agreed entirely that sending struct padding across the net is less than optimal :)
        saagarjha 7y ago
        > You have to assume that data is malicious, and will attack any weakness. The point is that the standard cannot help you here because malicious data attacks things outside the scope of the standard.
        olliej 7y ago
        The standard gives compilers leeway to handle “undefined” behaviour however they want. The original purpose of “undefined behaviour” was to deal with the myriad different platform behaviours of the era (remember at that point in history 1 byte was not necessary 8 bits). At some point the compiler writers decided to extend the effect of that to “if this behaviour is undefined in the spec” then we are allowed to treat the code as doing anything at all - so we went from this may do different things on different platforms to “the compiler can emit code that does something other than the clear intent or the platform specific behaviour”. This change in mindset from the compiler devs has been necessary to improve performance in the spec benchmarks, and other micro-benchmarks created primarily to justify the arbitrary definition of “undefined behaviour”. To give an idea of how capricious the compiler devs are they implemented optimizations that ended up breaking spec (it has undefined behavior), and so rolled those optimizations out. Yet when they literally add security bugs to code that has been secure for a decade, their response has been “well you’re relying on undefined behaviour”. The whole nonsense can be summed up as a (according to the insane definition of “correct” used by the compiler writers) Struct {char c;int i;} a,b; Bzero(&a, size of a) Bzero(&b, size of b) If (something) return memcmp(&Zaki, &b, size of a); Return 3; Could legitimately be optimized to a function that just returns 3. Because the writes to padding in bzero aren’t uneeefined, as are the reads in memcmp. Therefore the memcmp is undefined, and the compiler can assume undefined behavior cannot occur, therefore that branch can be removed. Whether the compiler does do that today isn’t relevant: the definition of “correct” used by compiler developers is that that is valid behaviour from them. The reality is that modern c/c++ developers need to treat the compiler as an adversary, because of a nonsensical definition of “correct code generation” assumed by the compiler writers.
  - olliej 7y ago
    I just realized you might be thinking of security in the sense of keys/partial keys. Things that matter: * vtable and code pointers: leaks the aslr slide * data pointers: leak heap location * type ids: can be used to spoof objects * length and size fields: can give you information about heap layout Etc etc. All of these get attacked.
    - saagarjha 7y ago
      The standard (and compilers that conform to it) have no sense of security baked into them. They don't know that they should wipe certain buffers because as far as it's concerned the C "abstract virtual machine" provides no way of accessing such data. Of course, as a real software engineer you know that you will make mistakes and these will frequently allow implementation details to leak in ways that are exploitable; however, there is no standards-compliant way to protect against this because as far as it is concerned you always write correct code. There are ways to influence the compiler to do certain things that help in the case of mistakes (memset_s, __attribute__((packed))) but these do not and cannot solve the underlying problem because they are by definition non-portable: they change things that are implementation-defined and not standards-defined.
- MichaelMoser123 7y ago
  are you sure that the padding is not zeroed by memset? [1] says that memset_s exists because it is not optimized away if it just clears a variable that will not be used after the call (unlike memset) On the other end: people who do this for security reasons say not to trust memset_s is 'an optional feature of c11 and not really portable' [2] [1] https://en.cppreference.com/w/c/string/byte/memset https://en.cppreference.com/w/c/string/byte/memset [2] https://www.cryptologie.net/article/419/zeroing-memory-compiler-optimizations-and-memset_s/ https://www.cryptologie.net/article/419/zeroing-memory-compi...
  - saagarjha 7y ago
    I don't think memset is required to zero out any padding, because the effects of this are not observable.
    - MichaelMoser123 7y ago
      The function memset gets the length of the area to overwrite, how is it to know where the padding is?
      - saagarjha 7y ago
        The compiler knows the layout of your structure and can choose to elide memset calls or replace them with equivalent code.
  - rurban 7y ago
    This [2] is a dangerous post from a security guy, describing memset_s with just a compiler barrier as "secure". In the age of Spectre only a memset_s with a memory barrier is secure, and currently all memset_s but my safeclib memset_s are insecure. Esp. the ones with "Secure" in its name.
  - deleted 7y ago
    [deleted]

Surely you only need memset_s if you're doing this for security reasons, as if memset is optimized away it's because the compiler can prove the observable semantics are the same.

olliej 7y ago

Nope, memset will often be “optimized” such that padding is not zeroed. It’s not because the compiler sees that padding isn’t observed, it’s because reading from padding is undefined. Because reading from padding is undefined the write (also undefined) by definition has no observable side effects. Memset_s is the only way to guarantee that padding is actually zeroed, well memset or bzero_s obviously. It’s critically important for people to understand that if the C spec says doing X is undefined, then current compiler writers interpret this as allowing them to do anything they want, even when it clearly introduces security vulnerabilities.

eridius 7y ago
Right, but zeroing padding only matters for security reasons. If your memset isn't being done for security reasons, then memset() is better than memset_s().
- olliej 7y ago
  If you are processing data from the network, any uninitialized read gets attacked. My experience is padding (both inside a struct, and outside - array alignment for instance) gets attacked. The only safe option is to initialize everything. I say this having spent more than a decade working on software that is always being attacked.
  - Matheus28 7y ago
    You shouldn't be sending structs over the network, since their specific layout depends on the compiler (and version). At least make them packed if you're gonna do undefined behavior... But you should just do it the right way and have a wrapper that deals with endianness differences and such: struct S my_struct; // ... push_int64(&output_stream, &my_struct.value1); push_string(&output_stream, my_struct.value2); push_int32(&output_stream, &my_struct.value3); And on the receiving side: struct S my_struct; next_int64(&input_stream, &my_struct.value1); next_string(&input_stream, my_struct.value2, sizeof(my_struct.value2)); next_int32(&input_stream, &my_struct.value3);
    - olliej 7y ago
      The problem is not sending structs across the network. The problem is processing any data from the network. You have to assume that data is malicious, and will attack any weakness. I’m not talking about serialisation at all - I agreed entirely that sending struct padding across the net is less than optimal :)
      - saagarjha 7y ago
        > You have to assume that data is malicious, and will attack any weakness. The point is that the standard cannot help you here because malicious data attacks things outside the scope of the standard.
        olliej 7y ago
        The standard gives compilers leeway to handle “undefined” behaviour however they want. The original purpose of “undefined behaviour” was to deal with the myriad different platform behaviours of the era (remember at that point in history 1 byte was not necessary 8 bits). At some point the compiler writers decided to extend the effect of that to “if this behaviour is undefined in the spec” then we are allowed to treat the code as doing anything at all - so we went from this may do different things on different platforms to “the compiler can emit code that does something other than the clear intent or the platform specific behaviour”. This change in mindset from the compiler devs has been necessary to improve performance in the spec benchmarks, and other micro-benchmarks created primarily to justify the arbitrary definition of “undefined behaviour”. To give an idea of how capricious the compiler devs are they implemented optimizations that ended up breaking spec (it has undefined behavior), and so rolled those optimizations out. Yet when they literally add security bugs to code that has been secure for a decade, their response has been “well you’re relying on undefined behaviour”. The whole nonsense can be summed up as a (according to the insane definition of “correct” used by the compiler writers) Struct {char c;int i;} a,b; Bzero(&a, size of a) Bzero(&b, size of b) If (something) return memcmp(&Zaki, &b, size of a); Return 3; Could legitimately be optimized to a function that just returns 3. Because the writes to padding in bzero aren’t uneeefined, as are the reads in memcmp. Therefore the memcmp is undefined, and the compiler can assume undefined behavior cannot occur, therefore that branch can be removed. Whether the compiler does do that today isn’t relevant: the definition of “correct” used by compiler developers is that that is valid behaviour from them. The reality is that modern c/c++ developers need to treat the compiler as an adversary, because of a nonsensical definition of “correct code generation” assumed by the compiler writers.
- olliej 7y ago
  I just realized you might be thinking of security in the sense of keys/partial keys. Things that matter: * vtable and code pointers: leaks the aslr slide * data pointers: leak heap location * type ids: can be used to spoof objects * length and size fields: can give you information about heap layout Etc etc. All of these get attacked.
  - saagarjha 7y ago
    The standard (and compilers that conform to it) have no sense of security baked into them. They don't know that they should wipe certain buffers because as far as it's concerned the C "abstract virtual machine" provides no way of accessing such data. Of course, as a real software engineer you know that you will make mistakes and these will frequently allow implementation details to leak in ways that are exploitable; however, there is no standards-compliant way to protect against this because as far as it is concerned you always write correct code. There are ways to influence the compiler to do certain things that help in the case of mistakes (memset_s, __attribute__((packed))) but these do not and cannot solve the underlying problem because they are by definition non-portable: they change things that are implementation-defined and not standards-defined.
MichaelMoser123 7y ago
are you sure that the padding is not zeroed by memset? [1] says that memset_s exists because it is not optimized away if it just clears a variable that will not be used after the call (unlike memset) On the other end: people who do this for security reasons say not to trust memset_s is 'an optional feature of c11 and not really portable' [2] [1] https://en.cppreference.com/w/c/string/byte/memset https://en.cppreference.com/w/c/string/byte/memset [2] https://www.cryptologie.net/article/419/zeroing-memory-compiler-optimizations-and-memset_s/ https://www.cryptologie.net/article/419/zeroing-memory-compi...
- saagarjha 7y ago
  I don't think memset is required to zero out any padding, because the effects of this are not observable.
  - MichaelMoser123 7y ago
    The function memset gets the length of the area to overwrite, how is it to know where the padding is?
    - saagarjha 7y ago
      The compiler knows the layout of your structure and can choose to elide memset calls or replace them with equivalent code.
- rurban 7y ago
  This [2] is a dangerous post from a security guy, describing memset_s with just a compiler barrier as "secure". In the age of Spectre only a memset_s with a memory barrier is secure, and currently all memset_s but my safeclib memset_s are insecure. Esp. the ones with "Secure" in its name.
- deleted 7y ago
  [deleted]