6 ms·

Hardening mode for the compiler

158 points by vitaut 1y ago

dilawar 1y ago
> So this mode needs to set user expectations appropriately: your code breaking between compiler releases is a feature, not a bug. Good luck. I feel that the C++ community values backward compatibility way too much for this to succeed. Most package maintainers are not going to like it a bit.
- pjmlp 1y ago
  There has been plenty of breakage throughout ISO revisions. The biggest problem is ABI, in theory that isn't something that standard cares about, in practice all compiler vendors do, thus proposals that break ABI from existing binary libraries tend to be an issue. Another issue is that WG21 nowadays is full of people without compiler experience, willing to push through their proposals, even without implementations, which then compiler vendors are supposed to suck it up and implement them somehow. After around C++14 time, it became cool to join WG21 and now the process is completely broken, there are more than 200 members. There is no guidance on an overall vision per se, everyone gets to submit their pet proposal, and then needs to champion it. Most of these folks aren't that keen into security, hence the kind of baby steps that have been happening.
  - dzaima 1y ago
    Compilers at least allow specifying the standard to target, which solves the ISO revision issue. But breaking within the same -std=... setting is quite a bit more annoying, forcing either indefinite patching on otherwise-complete functional codebases, or keeping potentially every compiler version on your system, both of which are pretty terrible options.
    - charcircuit 1y ago
      Assuming the code is position independent why can't the linker translate the ABI?
      - tempodox 1y ago
        Data sizes, alignment, the way stuff is loaded into registers, all that can change.
        tialaramex 1y ago
        My favourite weird ABI choice is: Who does what for a barrier? The barrier needs one party to do work for a full fence, but which party that should be doesn't matter... There can be an x86 FENCE over on the store side, or we can put the x86 FENCE on the load. We could do both but that's pointless, so don't do that. However if we do neither we haven't built a barrier and now there's probably a horrible bug.
      - dzaima 1y ago
        Maybe some things could be translated by a linker, but a linker can't change the size/layout of an in-memory data structure, and there's no info on what to translate from, even if info was added on what to translate to, anyway.
      - KingLancelot 1y ago
        [dead]
    - pjmlp 1y ago
      Breaking within the same std, is something impossible to prevent in compiled languages with enough freedom in build. Even the C ABI many talk about, most of them don't have any idea of what they are actually talking about. First of all, it is the OS ABI, in operating systems that happened to be written in C. Secondly, even C binary libraries have plenty of breakage opportunities within the same std, and compiler. ABI stability even in languages that kind of promise it, is in reality an half promise. Bytecode, or some part of the language is guaranteed to be stable, while being tied to a specific version, not all build flags fall under the promise, and not much is promised over the standard library. Even other good examples that go to great efforts like Java, .NET or Swift, aren't fully ABI safe.
      - yjftsjthsd-h 1y ago
        > First of all, it is the OS ABI, in operating systems that happened to be written in C. It may be per-OS (I wouldn't try linking Linux and NT object files even if they were both compiled from C by GCC with matching versions and everything), but enough details come from C that I think it's fair to call it a C ABI. Like, I can write unix software in pascal, but in order to write to stdout that code is gonna have to convert pascal strings into C strings. OTOH, pascal binaries using pascal libraries can use pascal semantics even on an OS that uses C ABIs.
        pjmlp 1y ago
        Strings is the easy part. Try to link two binary libraries in Linux, both compiled with GCC, while not using exactly the same compiler flags, or the same data padding, for example things like structures. Since committee people can explain it even better, "To Save C, We Must Save ABI" https://thephd.dev/to-save-c-we-must-save-abi-fixing-c-function-abi https://thephd.dev/to-save-c-we-must-save-abi-fixing-c-funct...
        uecker 1y ago
        Sorry, this is nonsense. Binaries link just fine on Linux. If you use a compiler flag that changes the ABI, then you are on your own, of course, but the GCC docu makes it very clear which specific flags those are. There is some corner cases with problems where you get problems if you use different compilers, e.g. atomic alignment (by adopting the broken C++ design into C) and some other corner cases where compilers did things slightly different.
    - dvtkrlbs 1y ago
      I wish the additional proposak that would add Eust like editions with the cpp moduled were expected. So sad it didnt pass.
- porridgeraisin 1y ago
  I don't like that statement (or that whole paragraph) one bit either. My packages breaking between compiler releases is most definitely a big fat bug. If bounds checks are going to be added, cool, -fstl-bounds-check. Or -fhardened like GCC. But not by default. Working existing code is working existing code, I don't care if it looks "suspicious" to some random guy's random compiler feature.
  - convolvatron 1y ago
    I'm kind of with you on the coding-style warning flags. it does really bother me that some opinionated person has decided that the way I use parenthesis needs to be punished. but I totally disagree with your second point. running code often has real problems with race conditions, error handling, unwanted memory reuse, out of bounds pointers, etc. if a new version of the compiler can prove these things for me - that's invaluable.
    - porridgeraisin 1y ago
      I too love those features. Just behind an option, so that existing scripts etc still continue to work. If many of those features are being added and the flags might add up to become a pain, then even a group flag -f-new-safety-features or whatever.
- thefaux 1y ago
  For many, backwards compatibility == long term employment.
wyldfire 1y ago
A really good accompaniment to this is Carruth's "C++, bounds checking, performance, and compilers" [1]: > ... strong belief that bounds checks couldn’t realistically be made cheap enough to enable by default. However, so far they are looking very affordable. From the above post, 0.3% for bounds checks in all the standard library types! There's more to the hardening story than just bounds checks. But it's a big part IMO. [1] https://chandlerc.blog/posts/2024/11/story-time-bounds-checking/ https://chandlerc.blog/posts/2024/11/story-time-bounds-check...
- tempodox 1y ago
  Even if bounds checks were only active in debug builds, that would already be of high value.
  - pjmlp 1y ago
    That at least has been covered almost since C++ exists. First in compiler vendors frameworks, pre C++98, afterwards with build settings. It is quite telling from existing community culture, that some folks only read their compiler manuals when government knocks on the door.
    - tester756 1y ago
      >It is quite telling from existing community culture, that some folks only read their compiler manuals when government knocks on the door. What do you want to say? Is this bad? I think this is desired. Only in c or c++ world people act like understanding how compiler internals work (often poorly) is desired
      - pjmlp 1y ago
        Where in the world reading a compiler manual means understanding compiler internals?!? One does not need to understand compiler internals to be aware what build flags are used to turn bounds checking on the standard library.
      - 1y ago
another_twist 1y ago
Maybe an easier way out is to add safe access instructions to LLVM itself. Its an IR after all, it should be possible to do a 3 phase update - add instructions to the IR, update the intermediate LLVM generator, then update the targetting backends.
ajb 1y ago
In the long term, it might be best to disable the ability to switch off checks using command line flags (which usually means, the whole executable) and only allow it on individual functions. Although the current mechanism to switch them off per function isn't idiot proof either (you need to remember to "#pragma diagnostic pop" after ) - we really need to be able to do it in a function attribute.
rurban 1y ago
They should also turn off the C11 Unicode identifier bugs with -fhardened, which enabled homoglyph attacks. There is no plan for C26 to fix this. No unicode identifiers without proper security measures
- rwmj 1y ago
  What is the threat profile here? I don't understand how this would be exploited in the real world. Once you're linking to a library, there are so many ways for the library to exploit your main program (eg. by running arbitrary code in constructors).
  - rurban 1y ago
    https://github.com/rurban/libu8ident https://github.com/rurban/libu8ident Search for homoglyph attacks and the unicode security guidelines for identifiers
    - rwmj 1y ago
      OK that is pretty interesting. For the TL;DR crowd, the exploit was: if(environmentǃ=ENV_PROD){ // bypass authZ checks in DEV return true; } where the 'ǃ' is a Unicode homoglyph (U+1C3 "LATIN LETTER ALVEOLAR CLICK") which obviously completely changes the nature of the code. I'll note that GCC gives a clear warning here ("suggest parentheses around assignment used as truth value"), so as always, turn on -Werror and take warnings seriously!
      - quuxplusone 1y ago
        The shown code is JavaScript; it wouldn't compile as C, because "environment[alveolar-click]" was never declared, and C requires declare-before-use. Does the advice to use GCC -Werror still apply to JavaScript? (I'd guess no, but I don't know for sure if I'm missing something.)
        rwmj 1y ago
hexagrams64 1y ago
[dead]

158 points by vitaut 1y ago

> So this mode needs to set user expectations appropriately: your code breaking between compiler releases is a feature, not a bug. Good luck. I feel that the C++ community values backward compatibility way too much for this to succeed. Most package maintainers are not going to like it a bit.

pjmlp 1y ago
There has been plenty of breakage throughout ISO revisions. The biggest problem is ABI, in theory that isn't something that standard cares about, in practice all compiler vendors do, thus proposals that break ABI from existing binary libraries tend to be an issue. Another issue is that WG21 nowadays is full of people without compiler experience, willing to push through their proposals, even without implementations, which then compiler vendors are supposed to suck it up and implement them somehow. After around C++14 time, it became cool to join WG21 and now the process is completely broken, there are more than 200 members. There is no guidance on an overall vision per se, everyone gets to submit their pet proposal, and then needs to champion it. Most of these folks aren't that keen into security, hence the kind of baby steps that have been happening.
- dzaima 1y ago
  Compilers at least allow specifying the standard to target, which solves the ISO revision issue. But breaking within the same -std=... setting is quite a bit more annoying, forcing either indefinite patching on otherwise-complete functional codebases, or keeping potentially every compiler version on your system, both of which are pretty terrible options.
  - charcircuit 1y ago
    Assuming the code is position independent why can't the linker translate the ABI?
    - tempodox 1y ago
      Data sizes, alignment, the way stuff is loaded into registers, all that can change.
      - tialaramex 1y ago
        My favourite weird ABI choice is: Who does what for a barrier? The barrier needs one party to do work for a full fence, but which party that should be doesn't matter... There can be an x86 FENCE over on the store side, or we can put the x86 FENCE on the load. We could do both but that's pointless, so don't do that. However if we do neither we haven't built a barrier and now there's probably a horrible bug.
    - dzaima 1y ago
      Maybe some things could be translated by a linker, but a linker can't change the size/layout of an in-memory data structure, and there's no info on what to translate from, even if info was added on what to translate to, anyway.
    - KingLancelot 1y ago
      [dead]
  - pjmlp 1y ago
    Breaking within the same std, is something impossible to prevent in compiled languages with enough freedom in build. Even the C ABI many talk about, most of them don't have any idea of what they are actually talking about. First of all, it is the OS ABI, in operating systems that happened to be written in C. Secondly, even C binary libraries have plenty of breakage opportunities within the same std, and compiler. ABI stability even in languages that kind of promise it, is in reality an half promise. Bytecode, or some part of the language is guaranteed to be stable, while being tied to a specific version, not all build flags fall under the promise, and not much is promised over the standard library. Even other good examples that go to great efforts like Java, .NET or Swift, aren't fully ABI safe.
    - yjftsjthsd-h 1y ago
      > First of all, it is the OS ABI, in operating systems that happened to be written in C. It may be per-OS (I wouldn't try linking Linux and NT object files even if they were both compiled from C by GCC with matching versions and everything), but enough details come from C that I think it's fair to call it a C ABI. Like, I can write unix software in pascal, but in order to write to stdout that code is gonna have to convert pascal strings into C strings. OTOH, pascal binaries using pascal libraries can use pascal semantics even on an OS that uses C ABIs.
      - pjmlp 1y ago
        Strings is the easy part. Try to link two binary libraries in Linux, both compiled with GCC, while not using exactly the same compiler flags, or the same data padding, for example things like structures. Since committee people can explain it even better, "To Save C, We Must Save ABI" https://thephd.dev/to-save-c-we-must-save-abi-fixing-c-function-abi https://thephd.dev/to-save-c-we-must-save-abi-fixing-c-funct...
        uecker 1y ago
        Sorry, this is nonsense. Binaries link just fine on Linux. If you use a compiler flag that changes the ABI, then you are on your own, of course, but the GCC docu makes it very clear which specific flags those are. There is some corner cases with problems where you get problems if you use different compilers, e.g. atomic alignment (by adopting the broken C++ design into C) and some other corner cases where compilers did things slightly different.
  - dvtkrlbs 1y ago
    I wish the additional proposak that would add Eust like editions with the cpp moduled were expected. So sad it didnt pass.
porridgeraisin 1y ago
I don't like that statement (or that whole paragraph) one bit either. My packages breaking between compiler releases is most definitely a big fat bug. If bounds checks are going to be added, cool, -fstl-bounds-check. Or -fhardened like GCC. But not by default. Working existing code is working existing code, I don't care if it looks "suspicious" to some random guy's random compiler feature.
- convolvatron 1y ago
  I'm kind of with you on the coding-style warning flags. it does really bother me that some opinionated person has decided that the way I use parenthesis needs to be punished. but I totally disagree with your second point. running code often has real problems with race conditions, error handling, unwanted memory reuse, out of bounds pointers, etc. if a new version of the compiler can prove these things for me - that's invaluable.
  - porridgeraisin 1y ago
    I too love those features. Just behind an option, so that existing scripts etc still continue to work. If many of those features are being added and the flags might add up to become a pain, then even a group flag -f-new-safety-features or whatever.
thefaux 1y ago
For many, backwards compatibility == long term employment.

wyldfire 1y ago

A really good accompaniment to this is Carruth's "C++, bounds checking, performance, and compilers" [1]: > ... strong belief that bounds checks couldn’t realistically be made cheap enough to enable by default. However, so far they are looking very affordable. From the above post, 0.3% for bounds checks in all the standard library types! There's more to the hardening story than just bounds checks. But it's a big part IMO. [1] https://chandlerc.blog/posts/2024/11/story-time-bounds-checking/ https://chandlerc.blog/posts/2024/11/story-time-bounds-check...

tempodox 1y ago
Even if bounds checks were only active in debug builds, that would already be of high value.
- pjmlp 1y ago
  That at least has been covered almost since C++ exists. First in compiler vendors frameworks, pre C++98, afterwards with build settings. It is quite telling from existing community culture, that some folks only read their compiler manuals when government knocks on the door.
  - tester756 1y ago
    >It is quite telling from existing community culture, that some folks only read their compiler manuals when government knocks on the door. What do you want to say? Is this bad? I think this is desired. Only in c or c++ world people act like understanding how compiler internals work (often poorly) is desired
    - pjmlp 1y ago
      Where in the world reading a compiler manual means understanding compiler internals?!? One does not need to understand compiler internals to be aware what build flags are used to turn bounds checking on the standard library.
    - 1y ago

another_twist 1y ago

Maybe an easier way out is to add safe access instructions to LLVM itself. Its an IR after all, it should be possible to do a 3 phase update - add instructions to the IR, update the intermediate LLVM generator, then update the targetting backends.

ajb 1y ago

In the long term, it might be best to disable the ability to switch off checks using command line flags (which usually means, the whole executable) and only allow it on individual functions. Although the current mechanism to switch them off per function isn't idiot proof either (you need to remember to "#pragma diagnostic pop" after ) - we really need to be able to do it in a function attribute.

rurban 1y ago

They should also turn off the C11 Unicode identifier bugs with -fhardened, which enabled homoglyph attacks. There is no plan for C26 to fix this. No unicode identifiers without proper security measures

rwmj 1y ago
What is the threat profile here? I don't understand how this would be exploited in the real world. Once you're linking to a library, there are so many ways for the library to exploit your main program (eg. by running arbitrary code in constructors).
- rurban 1y ago
  https://github.com/rurban/libu8ident https://github.com/rurban/libu8ident Search for homoglyph attacks and the unicode security guidelines for identifiers
  - rwmj 1y ago
    OK that is pretty interesting. For the TL;DR crowd, the exploit was: if(environmentǃ=ENV_PROD){ // bypass authZ checks in DEV return true; } where the 'ǃ' is a Unicode homoglyph (U+1C3 "LATIN LETTER ALVEOLAR CLICK") which obviously completely changes the nature of the code. I'll note that GCC gives a clear warning here ("suggest parentheses around assignment used as truth value"), so as always, turn on -Werror and take warnings seriously!
    - quuxplusone 1y ago
      The shown code is JavaScript; it wouldn't compile as C, because "environment[alveolar-click]" was never declared, and C requires declare-before-use. Does the advice to use GCC -Werror still apply to JavaScript? (I'd guess no, but I don't know for sure if I'm missing something.)
      - rwmj 1y ago

hexagrams64 1y ago

[dead]