5 ms·

memset is not a special case. Compiler designers are very much against special casing, and rightfully so. Compilers do have an intrinsic equivalent but that ha

memset is not a special case. Compiler designers are very much against special casing, and rightfully so. Compilers do have an intrinsic equivalent but that has nothing to do with whether it gets optimized out or kept. Following SSA rules, if you are writing to a non-volatile memory and not reading from it for the remaining duration of its lifetime then it's dead code. use memset_s was a suggestion if your goal is to scrub memory because it guarantees the write will happen. The guarantee comes from the C standard: > Unlike memset, any call to the memset_s function shall be evaluated strictly according to the rules of the abstract machine as described in (5.1.2.3). That is, any call to the memset_s function shall assume that the memory indicated by s and n may be accessible in the future and thus must contain the values indicated by c Which essentially means treat the memory as-if it's volatile.

1ris 5y ago
Are you allowed to cast the void * to volatile void *, pass that to memset and except the same result, or is the void *, volatile void*, void * conversion eliminated aswell?
- ncmncm 5y ago
  You can wave whichever dead chickens you like over the keyboard, all to the same effect. The compiler ensures only semantics that are observable. If you pass a pointer to a volatile or atomic object to another function the compiler cannot see into, the compiler knows that any specified operations might be observable, so must act accordingly. Then, operations on that object will not be optimized away. Or, anyway, not until someone builds the program using Link Time Optimization ("-flto") and the called function body becomes visible.
  - 1ris 5y ago
    I'm confused. Is the memset to the temporary volatile void* observable or not, and is the temporary possibly eliminated?
    - ncmncm 5y ago
      Casting the pointer passed to memset has no effect. The compiler knows what the storage pointed-to is: a temporary object. The compiler knows that memset does not allow a pointer to the object to escape from view. So, it can eliminate the call. Calling C11's memset_s, or MS's SecureZeroMemory, or FreeBSD's explicit_bzero, has the effect, to the compiler, of making the object observable, so that operations on it will not be optimized away. There is no way to express this locally within the language; you must rely on facilities provided by a Standard or your platform, or on the dodgy linkage trick cited elsewhere.
- halayli 5y ago
  volatile is a type qualifier and can be casted. However, memset's declaration takes a void* which prevents you from passing a volatile void*. casting or not won't change anything in this case because what matters is what memset takes and not what the pointer's fully qualified type is prior to calling memset. From the compiler's perspective, you're passing a non-volatile pointer to memset which makes it a candidate for elimination if the pointer's lifetime ends after memset call.
- josephcsible 5y ago
  As of C17, adding a volatile qualifier like that doesn't do anything useful. The standard currently only requires that accesses to volatile objects be handled specially, not accesses to nonvolatile objects through volatile-qualified pointers. A lot of people, including Linus Torvalds [1], are unhappy that this is how it works, and it looks like DR 476 may finally change it in C2x. [1]: https://lore.kernel.org/lkml/Pine.LNX.4.64.0607060856080.12404@g5.osdl.org/ https://lore.kernel.org/lkml/Pine.LNX.4.64.0607060856080.124...
ncmncm 5y ago
Treating "the memory as-if it's volatile" would not block such an optimization. Specifically, zeroing a volatile or atomic that lives on the stack immediately before returning from a function whose scope it lives in is freely optimized away, because it is not observable. Many people mentally model atomic as akin to volatile, about which they also frequently harbor superstitions. I knew a FreeBSD core committer who insisted (and convinced our boss!) that volatile was entirely sufficient to serialize interactions between threads. Someone else expected a line with a volatile or atomic operation never to be optimized away, and was appalled that it happened. Assignment to a volatile stack object whose address has not escaped and is not used will, in fact, be optimized away, regardless of the "clear language of the Standard". Often, not performing what would appear to be a pointless optimization would block a more substantial optimization. Compiler writers optimize aggressively in order to break through to the substantial ones, which are most usually dead-code elimination that can then propagate backward and upward. If you really need for a volatile object and operations on it not to be optimized away, you need to allow a pointer to it to escape to somewhere the compiler cannot see. Have a function in a separate ".o" file, and pass the pointer to that. Or, better, use a compiler intrinsic or other facility provided specifically for the purpose, such as memset_s or explicit_bzero.
- josephcsible 5y ago
  > Specifically, zeroing a volatile [...] that lives on the stack immediately before returning from a function whose scope it lives in is freely optimized away > Assignment to a volatile stack object whose address has not escaped and is not used will, in fact, be optimized away I don't believe either of these statements. https://godbolt.org/z/8cqEdY9oE https://godbolt.org/z/8cqEdY9oE is a simple case where this doesn't happen. Can you post a Godbolt link or something that demonstrates when it does? Keep in mind that the standard says "Accesses to volatile objects are evaluated strictly according to the rules of the abstract machine." In particular, it doesn't say that accesses to non-volatile objects through volatile-qualified pointers will be handled in any special way. (This looks like it's going to be changed in C2x, but it hasn't been changed as of C17.) Also, it says "If an attempt is made to refer to an object defined with a volatile-qualified type through use of an lvalue with non-volatile-qualified type, the behavior is undefined."
overgard 5y ago
The default "memset" should just do what the programmer intended, and there should be a "memset_kinda_maybe" for the current interpretation. I think it's totally busted that users need to guess what way the compiler is going to maul their program. 99% of C/C++ programmers are not bored language lawyers with time to memorize thousand page specs. They use the language because they're pragmatic. Messing up peoples code with incredibly over-eager optimization is the opposite of pragmatism. I'd rather spend a day writing inline assembly for a hot path than two days dealing with the compiler breaking my code and making users data insecure becuase "usually" its ok to ignore a memset.
- kaba0 5y ago
  Frankly, just learn the language properly… Compiler writers can’t throw their hands up in the air and do some non-deterministic/non-specified thingy. That would be terrible.
  - overgard 5y ago
    I know the language perfectly fine; that doesn't stop me from recognizing how broken this all is. I can dance around these horrible land mines, I'm just saying take the landmines out because people are getting hurt.

1ris 5y ago

Are you allowed to cast the void * to volatile void *, pass that to memset and except the same result, or is the void *, volatile void*, void * conversion eliminated aswell?

ncmncm 5y ago
You can wave whichever dead chickens you like over the keyboard, all to the same effect. The compiler ensures only semantics that are observable. If you pass a pointer to a volatile or atomic object to another function the compiler cannot see into, the compiler knows that any specified operations might be observable, so must act accordingly. Then, operations on that object will not be optimized away. Or, anyway, not until someone builds the program using Link Time Optimization ("-flto") and the called function body becomes visible.
- 1ris 5y ago
  I'm confused. Is the memset to the temporary volatile void* observable or not, and is the temporary possibly eliminated?
  - ncmncm 5y ago
    Casting the pointer passed to memset has no effect. The compiler knows what the storage pointed-to is: a temporary object. The compiler knows that memset does not allow a pointer to the object to escape from view. So, it can eliminate the call. Calling C11's memset_s, or MS's SecureZeroMemory, or FreeBSD's explicit_bzero, has the effect, to the compiler, of making the object observable, so that operations on it will not be optimized away. There is no way to express this locally within the language; you must rely on facilities provided by a Standard or your platform, or on the dodgy linkage trick cited elsewhere.
halayli 5y ago
volatile is a type qualifier and can be casted. However, memset's declaration takes a void* which prevents you from passing a volatile void*. casting or not won't change anything in this case because what matters is what memset takes and not what the pointer's fully qualified type is prior to calling memset. From the compiler's perspective, you're passing a non-volatile pointer to memset which makes it a candidate for elimination if the pointer's lifetime ends after memset call.
josephcsible 5y ago
As of C17, adding a volatile qualifier like that doesn't do anything useful. The standard currently only requires that accesses to volatile objects be handled specially, not accesses to nonvolatile objects through volatile-qualified pointers. A lot of people, including Linus Torvalds [1], are unhappy that this is how it works, and it looks like DR 476 may finally change it in C2x. [1]: https://lore.kernel.org/lkml/Pine.LNX.4.64.0607060856080.12404@g5.osdl.org/ https://lore.kernel.org/lkml/Pine.LNX.4.64.0607060856080.124...

ncmncm 5y ago

Treating "the memory as-if it's volatile" would not block such an optimization. Specifically, zeroing a volatile or atomic that lives on the stack immediately before returning from a function whose scope it lives in is freely optimized away, because it is not observable. Many people mentally model atomic as akin to volatile, about which they also frequently harbor superstitions. I knew a FreeBSD core committer who insisted (and convinced our boss!) that volatile was entirely sufficient to serialize interactions between threads. Someone else expected a line with a volatile or atomic operation never to be optimized away, and was appalled that it happened. Assignment to a volatile stack object whose address has not escaped and is not used will, in fact, be optimized away, regardless of the "clear language of the Standard". Often, not performing what would appear to be a pointless optimization would block a more substantial optimization. Compiler writers optimize aggressively in order to break through to the substantial ones, which are most usually dead-code elimination that can then propagate backward and upward. If you really need for a volatile object and operations on it not to be optimized away, you need to allow a pointer to it to escape to somewhere the compiler cannot see. Have a function in a separate ".o" file, and pass the pointer to that. Or, better, use a compiler intrinsic or other facility provided specifically for the purpose, such as memset_s or explicit_bzero.

josephcsible 5y ago
> Specifically, zeroing a volatile [...] that lives on the stack immediately before returning from a function whose scope it lives in is freely optimized away > Assignment to a volatile stack object whose address has not escaped and is not used will, in fact, be optimized away I don't believe either of these statements. https://godbolt.org/z/8cqEdY9oE https://godbolt.org/z/8cqEdY9oE is a simple case where this doesn't happen. Can you post a Godbolt link or something that demonstrates when it does? Keep in mind that the standard says "Accesses to volatile objects are evaluated strictly according to the rules of the abstract machine." In particular, it doesn't say that accesses to non-volatile objects through volatile-qualified pointers will be handled in any special way. (This looks like it's going to be changed in C2x, but it hasn't been changed as of C17.) Also, it says "If an attempt is made to refer to an object defined with a volatile-qualified type through use of an lvalue with non-volatile-qualified type, the behavior is undefined."

overgard 5y ago

The default "memset" should just do what the programmer intended, and there should be a "memset_kinda_maybe" for the current interpretation. I think it's totally busted that users need to guess what way the compiler is going to maul their program. 99% of C/C++ programmers are not bored language lawyers with time to memorize thousand page specs. They use the language because they're pragmatic. Messing up peoples code with incredibly over-eager optimization is the opposite of pragmatism. I'd rather spend a day writing inline assembly for a hot path than two days dealing with the compiler breaking my code and making users data insecure becuase "usually" its ok to ignore a memset.

kaba0 5y ago
Frankly, just learn the language properly… Compiler writers can’t throw their hands up in the air and do some non-deterministic/non-specified thingy. That would be terrible.
- overgard 5y ago
  I know the language perfectly fine; that doesn't stop me from recognizing how broken this all is. I can dance around these horrible land mines, I'm just saying take the landmines out because people are getting hurt.