3 ms·

Programs that would crash when using the memory-safe compiler aren't standards conforming. If you're worried that programs crashing due to bugs can be used for

Programs that would crash when using the memory-safe compiler aren't standards conforming. If you're worried that programs crashing due to bugs can be used for a denial-of-service attack... Well, yes, that is a thing. Low-level OS and device-handling code may need to do something that won't be seen as memory safe, but I expect that for such cases you'd need to do something similarly unsafe (eg, call an assembly-language routine) in any "memory safe" language. I'm not familiar with how ASAN is implemented, but since it doesn't change the number of bytes in a pointer variable, I expect that it either doesn't catch all out-of-bounds accesses or has a much higher (worst case) performance impact than what I outlined.

AlotOfReading 5y ago
I brought up ASAN because it's a real thing that already exists and gets run regularly. The broad details of how ASAN is implemented are best summarized in the original paper [1]. The practical short of it is that there are essentially no false negatives in anything remotely approaching real-world use. A malicious attacker could get around it, but any "better algorithm" would still run into the underlying issue that C doesn't have a way to actually handle detected unsafety and no amount of compiler magic will resolve that. You have to change the code. Whether that's by using another language annotations or through annotations like checked C is an interesting (but separate) discussion in its own right. As for the point that programs with memory unsafety aren't standards conforming; correct but irrelevant. Every nontrivial C program ever written is nonconformant. It's not a matter of "just write better code" at this point. [1] https://www.usenix.org/system/files/conference/atc12/atc12-final39.pdf https://www.usenix.org/system/files/conference/atc12/atc12-f...
- radford-neal 5y ago
  From the linked ASAN paper: "...at the relatively low cost of 73% slowdown and 3.4x increased memory usage..." That's too big a performance hit for production use - much bigger than you would get with the approach I outlined. I don't agree that any nontrivial C program is nonconformant, at least if you're talking about nonconformance due to invalid memory references. Referencing invalid memory locations is not the sort of thing that good programmers tolerate. (Of course, such references may occur when there are bugs - that's the reason for the run-time check - but not when a well-written program is operating as intended.)
  - nickelpro 5y ago
    That's a low cost for detecting the memory unsafe behvior. It is not intended to run in production, it's intended to run with your test suite.
    - radford-neal 5y ago
      Yes, I know. But this thread is about detecting invalid memory references in production, to prevent security exploits. ASAN seems too slow to solve that problem.
  - AlotOfReading 5y ago
    I usually find it safe to assume that compiler folks are conscious of optimization opportunities and make pretty intelligent tradeoffs on that spectrum. This is one such case. There's a long history of bounds checking compilers. The first that I know of is bcc in the 80s, which had a 10x slowdown! Austin et al. [1] came along a few years later way back in '94 and improved things to a mere 2-5x slowdown. That's pretty much where things stood for the next two decades because pointer accesses are everywhere in C and register pressure is nothing to sneeze at. Moreover, changing pointer sizes breaks your ability to link external things that weren't compiled with the same flags, like the system libc. ABI compatibility is make-or-break for a C compiler. You can get around that by breaking up the metadata from the actual pointer (e.g. softbound), but the performance cost is still ~3-4x [2]. ASAN was notable because 1) it was very efficient. That initial 73% was utterly fantastic at the time. 2) It was production-usable (i.e. worked on big codebases) 3) With hardware support, the performance hit is often under 10%. HWAsan on modern platforms is low-cost enough to run it all the time. And no, I'm saying that pretty much every nontrivial C program has UB, not that they're specifically memory unsafe. [1] https://minds.wisconsin.edu/bitstream/handle/1793/59822/TR1197.pdf?sequence=1 https://minds.wisconsin.edu/bitstream/handle/1793/59822/TR11... [2] https://insights.sei.cmu.edu/blog/performance-of-compiler-assisted-memory-safety-checking/ https://insights.sei.cmu.edu/blog/performance-of-compiler-as...
  - kaba0 5y ago
    With all due respect, why do you assume that your “thought about it 3 mins straight” idea would perform better than one that has been in the works for a long time now by people working on similar topics all of their lives? Don’t get me wrong, I often fell into this as well, but I think programmers really should get a bit of an ego-check sometimes, because (not you) it often affect discussions in other fields as well where we don’t know jackshit about.

AlotOfReading 5y ago

I brought up ASAN because it's a real thing that already exists and gets run regularly. The broad details of how ASAN is implemented are best summarized in the original paper [1]. The practical short of it is that there are essentially no false negatives in anything remotely approaching real-world use. A malicious attacker could get around it, but any "better algorithm" would still run into the underlying issue that C doesn't have a way to actually handle detected unsafety and no amount of compiler magic will resolve that. You have to change the code. Whether that's by using another language annotations or through annotations like checked C is an interesting (but separate) discussion in its own right. As for the point that programs with memory unsafety aren't standards conforming; correct but irrelevant. Every nontrivial C program ever written is nonconformant. It's not a matter of "just write better code" at this point. [1] https://www.usenix.org/system/files/conference/atc12/atc12-final39.pdf https://www.usenix.org/system/files/conference/atc12/atc12-f...

radford-neal 5y ago
From the linked ASAN paper: "...at the relatively low cost of 73% slowdown and 3.4x increased memory usage..." That's too big a performance hit for production use - much bigger than you would get with the approach I outlined. I don't agree that any nontrivial C program is nonconformant, at least if you're talking about nonconformance due to invalid memory references. Referencing invalid memory locations is not the sort of thing that good programmers tolerate. (Of course, such references may occur when there are bugs - that's the reason for the run-time check - but not when a well-written program is operating as intended.)
- nickelpro 5y ago
  That's a low cost for detecting the memory unsafe behvior. It is not intended to run in production, it's intended to run with your test suite.
  - radford-neal 5y ago
    Yes, I know. But this thread is about detecting invalid memory references in production, to prevent security exploits. ASAN seems too slow to solve that problem.
- AlotOfReading 5y ago
  I usually find it safe to assume that compiler folks are conscious of optimization opportunities and make pretty intelligent tradeoffs on that spectrum. This is one such case. There's a long history of bounds checking compilers. The first that I know of is bcc in the 80s, which had a 10x slowdown! Austin et al. [1] came along a few years later way back in '94 and improved things to a mere 2-5x slowdown. That's pretty much where things stood for the next two decades because pointer accesses are everywhere in C and register pressure is nothing to sneeze at. Moreover, changing pointer sizes breaks your ability to link external things that weren't compiled with the same flags, like the system libc. ABI compatibility is make-or-break for a C compiler. You can get around that by breaking up the metadata from the actual pointer (e.g. softbound), but the performance cost is still ~3-4x [2]. ASAN was notable because 1) it was very efficient. That initial 73% was utterly fantastic at the time. 2) It was production-usable (i.e. worked on big codebases) 3) With hardware support, the performance hit is often under 10%. HWAsan on modern platforms is low-cost enough to run it all the time. And no, I'm saying that pretty much every nontrivial C program has UB, not that they're specifically memory unsafe. [1] https://minds.wisconsin.edu/bitstream/handle/1793/59822/TR1197.pdf?sequence=1 https://minds.wisconsin.edu/bitstream/handle/1793/59822/TR11... [2] https://insights.sei.cmu.edu/blog/performance-of-compiler-assisted-memory-safety-checking/ https://insights.sei.cmu.edu/blog/performance-of-compiler-as...
- kaba0 5y ago
  With all due respect, why do you assume that your “thought about it 3 mins straight” idea would perform better than one that has been in the works for a long time now by people working on similar topics all of their lives? Don’t get me wrong, I often fell into this as well, but I think programmers really should get a bit of an ego-check sometimes, because (not you) it often affect discussions in other fields as well where we don’t know jackshit about.