4 ms·

I've done security auditing of ADA and C avionics code. (including for DAL-A components). As a language, I'd take Rust any day (in terms of security/robustness

I've done security auditing of ADA and C avionics code. (including for DAL-A components). As a language, I'd take Rust any day (in terms of security/robustness properties). However, the rust/LLVM compiler pipeline is nowhere near mature enough for use in high-criticality environments.

deleted 7y ago
[deleted]
winter_blue 7y ago
> However, the rust/LLVM compiler pipeline is nowhere near mature enough for use in high-criticality environments. I don't know about Rust, but LLVM isn't mature? Nearly ever program running in the Apple ecosystem was compiled using LLVM. Swift is compiled using LLVM. Since Xcode 4.2, Clang is the default compiler. So iOS and macOS apps are built with LLVM. I'd also wager a guess that Apple uses Clang to compile key system code (Darwin, macOS, and iOS, etc) as well.
- nicoburns 7y ago
  Not mission-critical, must-never-fail mature, no. All of these systems contain tens (if not thousands of bugs), some of these probably come from compiler issues sometimes.
- tonfa 7y ago
  I don't think consumer OSes are considered highly critical. (Unlike code running for avionics for example)
- shakna 7y ago
  LLVM is still in disagreement as to whether or not infinite loops are UB [1], and others. It is becoming more mature... But the argument about how to handle an empty infinite loop has been going on since 2015. [1] https://bugs.llvm.org/show_bug.cgi?id=24078 https://bugs.llvm.org/show_bug.cgi?id=24078
  - Gibbon1 7y ago
    Offhand thought of the day. The Motor Industry Software Reliability Association should ban compilers that allow optimization based in UB. Because seriously these people need to be stopped.
    - astrange 7y ago
      UB was invented for the purpose of optimizing. No UB, no optimizing. And nobody is interested in funding or working on compilers that don't optimize. Anyway, just test with ubsan and you'll be good.
      - dodobirdlord 7y ago
        At least a lot of it was due to having to specify around the weird quirks of existing compiler implementations. C has far more undefined behavior than it needs for optimization, and modern programming language theory and compiler research has shown that most undefined behavior isn't really necessary at all. [0] For example, safe Rust has extremely close (and sometimes superior) performance to C, and has almost* no undefined behavior at all. [1] [0] https://www.cs.utah.edu/~regehr/papers/undef-pldi17.pdf https://www.cs.utah.edu/~regehr/papers/undef-pldi17.pdf [1] https://benchmarksgame-team.pages.debian.net/benchmarksgame/fastest/gcc-rust.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/... * Some is inherited from LLVM, some is bugs, some is dumb corner cases like using kernel syscalls to overwrite the program's own memory with random bits.
      - pingyong 7y ago
        Name one example where UB is necessary for useful optimizations where simply having unspecified values or platform dependent behavior wouldn't be enough.
- hexane360 7y ago
  Another factor is that Rust is testing LLVM's codepaths in ways that Clang rarely does. A lot of bugs in LLVM have been uncovered through Rust.
- johannes1234321 7y ago
  MacOS and related don't run the airplane. Pilots have an iPad for some checklists and other things where "have you tried turning it off and on again" works as failure prcedure. If a failure, caused by a compiler bug leading to an overflow in a rare flight situation, happens on an airplane system a reboot is not really the option you want to depend on.
  - jasonjayr 7y ago
    Ironically, I recall reading about a mid-flight incident where the procedure to bring things back involved turning the entire plane off and on again.
    - johannes1234321 7y ago
      That might work in a stable situation - many (not all) planes are good gliders. (I.e. somewhere I read that the A380 is a quite good glider, among other reasons since the wings were designed for larger versions; an F-16 however won't really glide) In a critical situation this is impossible, though.
    - sandos 7y ago
      That will not always work very well, inertial navigation systems often wants to be completely still for a couple of minutes to calibrate themselves. I have heard about just flying with no G-load for a while and getting some basic instruments working. But I dont think you will get full nav functionality back.
  - adwww 7y ago
    I write software for in cockpit ipad apps - they are becoming more and more integral to the workload of pilots, everything from route planning, guidance, checklists, manifests and ordering fuel. There was actually an incident posted on HN a while ago where IIRC an ipad did a software update or similar at a critical moment, and caused complications bad enough to get an incident report published on some aviation agency's website.
btown 7y ago
> However, the rust/LLVM compiler pipeline is nowhere near mature enough for use in high-criticality environments. I love Rust, but I wouldn't want it to fly a plane I'm on. For instance, here's a bug in LLVM that Rust developers happened to discover - before they disabled their use of the buggy features, Rust/LLVM was producing _numerically incorrect_ code. https://github.com/rust-lang/rust/issues/54878#issuecomment-429578187 https://github.com/rust-lang/rust/issues/54878#issuecomment-...
- monocasa 7y ago
  The Ada compilers have their own cute bugs like this too. That's what happens with a niche language with relatively few users.
- bsder 7y ago
  Um, if you read the bug report, gcc also had the same bug. Age doesn't automatically protect you from bugs.
adrianN 7y ago
I'd take a compiler backend that compiles large parts of the modern software ecosystem every day and has countless programmers using it over some niche compiler that is used only for a vanishing fraction of all applications but boasts some kind of official stamp of applicability in safety relevant contexts.
- dodobirdlord 7y ago
  I'm not so sure I would. For one, Ada is in no way a niche language. For two, C (and C++ by extension) simply were not designed with safety or reliability in mind. I happen to think that it's fair to say that they weren't designed at all, in that C compilers existed before the first language specification, and the language specification was forced to accommodate their quirks. Cargo culting about C is fine and all, but it often leads to absurd claims that it is more suitable in any arbitrary domain than languages and compilers designed specifically for the domain. Behold, LLVM and GCC both miscompile the same numerical code involving mildly tricky aliasing constraints. https://github.com/rust-lang/rust/issues/54878#issuecomment-429578187 https://github.com/rust-lang/rust/issues/54878#issuecomment-... I think I'd prefer that safety-critical code be written in languages that don't allow pointer arithmetic except in scenarios that can be proved via static analysis not to introduce multiple memory aliases. Expecting program writers and compiler authors to get these sorts of things right in C/C++ is just unreasonable.
  - adrianN 7y ago
    You don't have to use C or C++ to use LLVM. I'd argue that you would probably never find a bug such as that in a certified compiler, not because of the certification, but because far, far fewer eyes look at its output. I have worked in safety critical software and used a certified compiler. It had enough bugs that I personally found four in my first year.

deleted 7y ago

[deleted]

winter_blue 7y ago

> However, the rust/LLVM compiler pipeline is nowhere near mature enough for use in high-criticality environments. I don't know about Rust, but LLVM isn't mature? Nearly ever program running in the Apple ecosystem was compiled using LLVM. Swift is compiled using LLVM. Since Xcode 4.2, Clang is the default compiler. So iOS and macOS apps are built with LLVM. I'd also wager a guess that Apple uses Clang to compile key system code (Darwin, macOS, and iOS, etc) as well.

nicoburns 7y ago
Not mission-critical, must-never-fail mature, no. All of these systems contain tens (if not thousands of bugs), some of these probably come from compiler issues sometimes.
tonfa 7y ago
I don't think consumer OSes are considered highly critical. (Unlike code running for avionics for example)
shakna 7y ago
LLVM is still in disagreement as to whether or not infinite loops are UB [1], and others. It is becoming more mature... But the argument about how to handle an empty infinite loop has been going on since 2015. [1] https://bugs.llvm.org/show_bug.cgi?id=24078 https://bugs.llvm.org/show_bug.cgi?id=24078
- Gibbon1 7y ago
  Offhand thought of the day. The Motor Industry Software Reliability Association should ban compilers that allow optimization based in UB. Because seriously these people need to be stopped.
  - astrange 7y ago
    UB was invented for the purpose of optimizing. No UB, no optimizing. And nobody is interested in funding or working on compilers that don't optimize. Anyway, just test with ubsan and you'll be good.
    - dodobirdlord 7y ago
      At least a lot of it was due to having to specify around the weird quirks of existing compiler implementations. C has far more undefined behavior than it needs for optimization, and modern programming language theory and compiler research has shown that most undefined behavior isn't really necessary at all. [0] For example, safe Rust has extremely close (and sometimes superior) performance to C, and has almost* no undefined behavior at all. [1] [0] https://www.cs.utah.edu/~regehr/papers/undef-pldi17.pdf https://www.cs.utah.edu/~regehr/papers/undef-pldi17.pdf [1] https://benchmarksgame-team.pages.debian.net/benchmarksgame/fastest/gcc-rust.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/... * Some is inherited from LLVM, some is bugs, some is dumb corner cases like using kernel syscalls to overwrite the program's own memory with random bits.
    - pingyong 7y ago
      Name one example where UB is necessary for useful optimizations where simply having unspecified values or platform dependent behavior wouldn't be enough.
hexane360 7y ago
Another factor is that Rust is testing LLVM's codepaths in ways that Clang rarely does. A lot of bugs in LLVM have been uncovered through Rust.
johannes1234321 7y ago
MacOS and related don't run the airplane. Pilots have an iPad for some checklists and other things where "have you tried turning it off and on again" works as failure prcedure. If a failure, caused by a compiler bug leading to an overflow in a rare flight situation, happens on an airplane system a reboot is not really the option you want to depend on.
- jasonjayr 7y ago
  Ironically, I recall reading about a mid-flight incident where the procedure to bring things back involved turning the entire plane off and on again.
  - johannes1234321 7y ago
    That might work in a stable situation - many (not all) planes are good gliders. (I.e. somewhere I read that the A380 is a quite good glider, among other reasons since the wings were designed for larger versions; an F-16 however won't really glide) In a critical situation this is impossible, though.
  - sandos 7y ago
    That will not always work very well, inertial navigation systems often wants to be completely still for a couple of minutes to calibrate themselves. I have heard about just flying with no G-load for a while and getting some basic instruments working. But I dont think you will get full nav functionality back.
- adwww 7y ago
  I write software for in cockpit ipad apps - they are becoming more and more integral to the workload of pilots, everything from route planning, guidance, checklists, manifests and ordering fuel. There was actually an incident posted on HN a while ago where IIRC an ipad did a software update or similar at a critical moment, and caused complications bad enough to get an incident report published on some aviation agency's website.

btown 7y ago

> However, the rust/LLVM compiler pipeline is nowhere near mature enough for use in high-criticality environments. I love Rust, but I wouldn't want it to fly a plane I'm on. For instance, here's a bug in LLVM that Rust developers happened to discover - before they disabled their use of the buggy features, Rust/LLVM was producing _numerically incorrect_ code. https://github.com/rust-lang/rust/issues/54878#issuecomment-429578187 https://github.com/rust-lang/rust/issues/54878#issuecomment-...

monocasa 7y ago
The Ada compilers have their own cute bugs like this too. That's what happens with a niche language with relatively few users.
bsder 7y ago
Um, if you read the bug report, gcc also had the same bug. Age doesn't automatically protect you from bugs.

adrianN 7y ago

I'd take a compiler backend that compiles large parts of the modern software ecosystem every day and has countless programmers using it over some niche compiler that is used only for a vanishing fraction of all applications but boasts some kind of official stamp of applicability in safety relevant contexts.

dodobirdlord 7y ago
I'm not so sure I would. For one, Ada is in no way a niche language. For two, C (and C++ by extension) simply were not designed with safety or reliability in mind. I happen to think that it's fair to say that they weren't designed at all, in that C compilers existed before the first language specification, and the language specification was forced to accommodate their quirks. Cargo culting about C is fine and all, but it often leads to absurd claims that it is more suitable in any arbitrary domain than languages and compilers designed specifically for the domain. Behold, LLVM and GCC both miscompile the same numerical code involving mildly tricky aliasing constraints. https://github.com/rust-lang/rust/issues/54878#issuecomment-429578187 https://github.com/rust-lang/rust/issues/54878#issuecomment-... I think I'd prefer that safety-critical code be written in languages that don't allow pointer arithmetic except in scenarios that can be proved via static analysis not to introduce multiple memory aliases. Expecting program writers and compiler authors to get these sorts of things right in C/C++ is just unreasonable.
- adrianN 7y ago
  You don't have to use C or C++ to use LLVM. I'd argue that you would probably never find a bug such as that in a certified compiler, not because of the certification, but because far, far fewer eyes look at its output. I have worked in safety critical software and used a certified compiler. It had enough bugs that I personally found four in my first year.