3 ms·

This post supports my personal experience. ASAN is a flippin' godsend. I have never encountered a false report from ASAN. I have seen it have false negatives bu

This post supports my personal experience. ASAN is a flippin' godsend. I have never encountered a false report from ASAN. I have seen it have false negatives but those are usually intermittent with true positives when the tests are run again and usually related to very small buffer overflows (eg, a "few bytes" past the end of a buffer). Not perfect but no false positives means it's 100% worthwhile to have enabled. But UBSAN is... not as helpful. It sometimes catches things. It often misses some undefined behavior that can be easily identified elsewhere. But the real kicker is that (almost?) all of the sanitizers are mutually exclusive. That really sucks if you're trying to debug something very complex. It also has lead me to build and drive the complexity into smaller and smaller chunks. Arguably that's a good thing! I have, for several years now, always turned Address Sanitizer on by default when -DCMAKE_BUILD_TYPE=Debug or off when it's not. I've added an option that forces the option with -DASAN_BUILD=bool. I wish Kitware would simplify my CMakeLists files by making this default behavior. And I think it would really help a lot of open source projects that aren't configured for ASAN too

cyber_kinetist 4y ago
I think there’s a place for Address Sanitizer to be enabled on RelWithDbgInfo build type (usually used for development and not production). Debug mode in C++ is already too slow, but adding a sanitizer makes it almost unusable for performance-sensitive code.
- kllrnohj 4y ago
  The sanitizers are not incompatible with optimizations. asan with -O2 is a perfectly valid & useful combination. It's much faster than valgrind or anything like that, very usable levels of performance. And on an increasingly common number of CPUs there's hardware acceleration for ASAN as well ( https://clang.llvm.org/docs/HardwareAssistedAddressSanitizerDesign.html https://clang.llvm.org/docs/HardwareAssistedAddressSanitizer... & https://source.android.com/docs/security/test/hwasan https://source.android.com/docs/security/test/hwasan )
mannykannot 4y ago
With regard to UBSAN, are the misses that you see examples of specific checkers failing to find the specific sort of UB that they are intended to detect? If so, do the majority of misses seem to come from a small subset of the checkers?
- inetknght 4y ago
  > are the misses that you see examples of specific checkers failing to find the specific sort of UB that they are intended to detect? I'm not sure, but I seem to recall that they were at least variants of intended checkers. It's honestly been about 5+ years since I last tried UBSan. I was that unimpressed with it. I do hope it's improved (though this blog post suggests improvements are minor). I would be much more likely to try a bunch of sanitizers again if they didn't interfere with my loved ASAN.
brandmeyer 4y ago
Running multiple test builds in parallel isn't all that difficult, though. One with ubsan, one with asan, and one (opt-in on a test-by-test basis) with tsan.
gpderetta 4y ago
The sanitizer are all great! I've also had TSAN find non-trivial issues with my threaded code. There is also the trick to declare the functions you want to test as constexpr functions and write your unit tests as static assertions to catch UB (well, modulo bugs in the constexpr support in the compiler of course).

cyber_kinetist 4y ago

I think there’s a place for Address Sanitizer to be enabled on RelWithDbgInfo build type (usually used for development and not production). Debug mode in C++ is already too slow, but adding a sanitizer makes it almost unusable for performance-sensitive code.

kllrnohj 4y ago
The sanitizers are not incompatible with optimizations. asan with -O2 is a perfectly valid & useful combination. It's much faster than valgrind or anything like that, very usable levels of performance. And on an increasingly common number of CPUs there's hardware acceleration for ASAN as well ( https://clang.llvm.org/docs/HardwareAssistedAddressSanitizerDesign.html https://clang.llvm.org/docs/HardwareAssistedAddressSanitizer... & https://source.android.com/docs/security/test/hwasan https://source.android.com/docs/security/test/hwasan )

mannykannot 4y ago

With regard to UBSAN, are the misses that you see examples of specific checkers failing to find the specific sort of UB that they are intended to detect? If so, do the majority of misses seem to come from a small subset of the checkers?

inetknght 4y ago
> are the misses that you see examples of specific checkers failing to find the specific sort of UB that they are intended to detect? I'm not sure, but I seem to recall that they were at least variants of intended checkers. It's honestly been about 5+ years since I last tried UBSan. I was that unimpressed with it. I do hope it's improved (though this blog post suggests improvements are minor). I would be much more likely to try a bunch of sanitizers again if they didn't interfere with my loved ASAN.

brandmeyer 4y ago

Running multiple test builds in parallel isn't all that difficult, though. One with ubsan, one with asan, and one (opt-in on a test-by-test basis) with tsan.

gpderetta 4y ago

The sanitizer are all great! I've also had TSAN find non-trivial issues with my threaded code. There is also the trick to declare the functions you want to test as constexpr functions and write your unit tests as static assertions to catch UB (well, modulo bugs in the constexpr support in the compiler of course).