5 ms·

> Issue 4: Windows kernel driver uses memset() to clear sensitive data > Calls to memset() run the risk of being optimized out by the compiler. I would be cur

by doe88 13y ago

> Issue 4: Windows kernel driver uses memset() to clear sensitive data > Calls to memset() run the risk of being optimized out by the compiler. I would be curious to know which compilers or which options actually still do these ill-fated optimizations?

cliffbean 13y ago
Pretty much all of them do. Many of the big leaders in the C++ world, like Google, are pushing the idea that every last optimization technique is crucially vital. Google can bank on tenths-of-a-percent of performance, so why can't you? Google can devote armadas of armadas of test machines to run asan/tsan/space-alien-san and everything else continuously, and it works pretty well for them, so why can't you do this too?
munin 13y ago
they still do, but everyone (including the windows DDK) offers a "secure memset" that casts to volatile and preserves the call. in Windows, it's called RtlSecureZeroMemory and I would think that know about this would be one of the pre-requisites for writing security software on that platform...
30thElement 13y ago
They should do it only in situations where it doesn't change the program behavior. I use memset frequently in C just to be safe, but if it's written to later on before it's ever read from, the compiler can optimize that away. I'm guessing their recommendation here is if you did something like char* plain_text = malloc(size); ///do stuff with plain_text memset(plain_text, 0, size); free(plain_text); For most programs that last memset is unnecessary (and may even be unnecessary according to the standard, but it's probably implementation defined, not undefined behavior) and it makes sense for the compiler to optimize it away. But for crypto purposes you have to be afraid of someone being able to read plain_text later, so the memset is important
- Marat_Dukhan 13y ago
  I checked 3 compilers: icc 14, gcc 4.8, and clang 3.3. Clang is the only one which optimized the memset away.
  - timtadh 13y ago
    Not a windows user so I can't check but given that it is the kernel driver the compiler we would be concerned about is the MS visual c++ compiler.
    - Marat_Dukhan 13y ago
      Visual C++ 2013 doesn't remove memset either.
  - lawnchair_larry 13y ago
    Did you enable optimization? I know first hand that GCC will optimize something similar out in many cases. http://gcc.gnu.org/bugzilla/show_bug.cgi?id=8537 http://gcc.gnu.org/bugzilla/show_bug.cgi?id=8537
    - Marat_Dukhan 13y ago
      Yes, I compiled with -O3.
shin_lao 13y ago
It is not advised to use C library functions. The driver should use RtlZeroMemory() and in that case RtlSecureZeroMemory(), but otherwise this optimization makes a lot of sense in many cases when you have for example an useless constructor call before an initialization.
doe88 13y ago
After a little bit of digging seems to be the best way to currently wipe out sensitive data is to implement something like in libsodium https://github.com/jedisct1/libsodium/blob/master/src/libsodium/sodium/utils.c#L19 https://github.com/jedisct1/libsodium/blob/master/src/libsod... 1- Try to use SecureZeroMemory() on Windows 2- Use memcpy_s() if available, which from its manpage is guaranteed not to be optimized away by compilers 3- Use a loop to manually zero-out the buffer after having casted to a volatile pointer, again to prevent compiler optimizations
lawnchair_larry 13y ago
All of them will under circumstances that tend to manifest when wiping secrets. This surprises a lot of developers, but it's a very common finding when auditing crypto code or other secret storage. It's typically done when you don't need the value anymore, of course. So, the compiler sees a write to memory that is never used. Which is exactly the type of thing compilers try to optimize out (if it can be proven that it is never used). From the C99 standard: In the abstract machine, all expressions are evaluated as specified by the semantics. An actual implementation need not evaluate part of an expression if it can deduce that its value is not used and that no needed side effects are produced (including any caused by calling a function or accessing a volatile object).

by doe88 13y ago

cliffbean 13y ago

Pretty much all of them do. Many of the big leaders in the C++ world, like Google, are pushing the idea that every last optimization technique is crucially vital. Google can bank on tenths-of-a-percent of performance, so why can't you? Google can devote armadas of armadas of test machines to run asan/tsan/space-alien-san and everything else continuously, and it works pretty well for them, so why can't you do this too?

munin 13y ago

they still do, but everyone (including the windows DDK) offers a "secure memset" that casts to volatile and preserves the call. in Windows, it's called RtlSecureZeroMemory and I would think that know about this would be one of the pre-requisites for writing security software on that platform...

30thElement 13y ago

They should do it only in situations where it doesn't change the program behavior. I use memset frequently in C just to be safe, but if it's written to later on before it's ever read from, the compiler can optimize that away. I'm guessing their recommendation here is if you did something like char* plain_text = malloc(size); ///do stuff with plain_text memset(plain_text, 0, size); free(plain_text); For most programs that last memset is unnecessary (and may even be unnecessary according to the standard, but it's probably implementation defined, not undefined behavior) and it makes sense for the compiler to optimize it away. But for crypto purposes you have to be afraid of someone being able to read plain_text later, so the memset is important

Marat_Dukhan 13y ago
I checked 3 compilers: icc 14, gcc 4.8, and clang 3.3. Clang is the only one which optimized the memset away.
- timtadh 13y ago
  Not a windows user so I can't check but given that it is the kernel driver the compiler we would be concerned about is the MS visual c++ compiler.
  - Marat_Dukhan 13y ago
    Visual C++ 2013 doesn't remove memset either.
- lawnchair_larry 13y ago
  Did you enable optimization? I know first hand that GCC will optimize something similar out in many cases. http://gcc.gnu.org/bugzilla/show_bug.cgi?id=8537 http://gcc.gnu.org/bugzilla/show_bug.cgi?id=8537
  - Marat_Dukhan 13y ago
    Yes, I compiled with -O3.

shin_lao 13y ago

It is not advised to use C library functions. The driver should use RtlZeroMemory() and in that case RtlSecureZeroMemory(), but otherwise this optimization makes a lot of sense in many cases when you have for example an useless constructor call before an initialization.

doe88 13y ago

After a little bit of digging seems to be the best way to currently wipe out sensitive data is to implement something like in libsodium https://github.com/jedisct1/libsodium/blob/master/src/libsodium/sodium/utils.c#L19 https://github.com/jedisct1/libsodium/blob/master/src/libsod... 1- Try to use SecureZeroMemory() on Windows 2- Use memcpy_s() if available, which from its manpage is guaranteed not to be optimized away by compilers 3- Use a loop to manually zero-out the buffer after having casted to a volatile pointer, again to prevent compiler optimizations

lawnchair_larry 13y ago

All of them will under circumstances that tend to manifest when wiping secrets. This surprises a lot of developers, but it's a very common finding when auditing crypto code or other secret storage. It's typically done when you don't need the value anymore, of course. So, the compiler sees a write to memory that is never used. Which is exactly the type of thing compilers try to optimize out (if it can be proven that it is never used). From the C99 standard: In the abstract machine, all expressions are evaluated as specified by the semantics. An actual implementation need not evaluate part of an expression if it can deduce that its value is not used and that no needed side effects are produced (including any caused by calling a function or accessing a volatile object).