3 ms·

Steve is too modest about the tests by the way. Lots of the conformance testing for Rust's compiler happens automatically. Herb mentions Perennial https://www.p

Steve is too modest about the tests by the way. Lots of the conformance testing for Rust's compiler happens automatically. Herb mentions Perennial https://www.peren.com/ https://www.peren.com/ and Plum Hall https://plumhall.com/ https://plumhall.com/ for C++ and Steve says "I'm not saying that functions as a conformance test suite" when it comes to Rust's own compiler tests. But, while the Rust project itself does not perform conformance testing, Ferrous Systems does and so yeah, analogous conformance test suites exist and are run automatically. So when Herb says "And in Rust's case, that's going to need to be built out" if the interview was taking place five years ago that's an important point. But it wasn't five years ago, it was only a few weeks ago.

myworkinisgood 2y ago
C++ leadership are blinded by their own success and have no information of outside world.
bregma 2y ago
> analogous conformance test suites exist and are run automatically I'm not clear on how this is even possible. "Conformance" for C++ means adherence to the published ISO/IEC 14882 standard. In the case of Rust, it's "this is what the compiler does this week". Sure, a third party has built out a CI system for Rust and uses it provide evidence that Rust does what Rust does, but that's nothing similar to having evidence any random toolchain conforms to an accepted, published, international standard of what the C++ language does. When it comes to providing test evidence for functional safety (for example ISO 26262 or IEC 61508) I can point to Perennial or Sold Sands results and say with confidence that my toolchain does what it says on the C++ box. When it comes to Rust, I can point to Ferrous and say "the core Rust language when built using rustc does what the rustc developers claimed it should do last week, excluding any crates or libraries." To which claim are you going to trust the lives of you and your loved ones? They aren't analogous at all. There is just reams of marketing spin here and eventually it's going to kill someone.
- jamesmunns 2y ago
  This is entirely incorrect, Ferrocene manages a specification of a subset of the language here: https://public-docs.ferrocene.dev/main/specification/index.html https://public-docs.ferrocene.dev/main/specification/index.h..., including things like a traceability matrix that links the specification to the implementation and the verification: https://public-docs.ferrocene.dev/main/qualification/traceability-matrix.html https://public-docs.ferrocene.dev/main/qualification/traceab... For each release of Ferrocene, this is kept up to date, and the same as C or C++, what is specified and stable can be relied upon, and implementation details are implementation details, the same as it would be if you switched from LLVM w/ SolidSands' SuperTest suite to IAR or GreenHills' toolchains which may have varying impl details but still maintain conformance with the specification. The majority of safety critical teams will snapshot a single toolchain for the entire development lifecycle (sometimes updating if necessary, very rarely), but Ferrocene is releasing updates that are approaching the full Rust cadence (IIRC they've discussed going to every other release, so once every 12 weeks vs Rust's 6 week cadence), with all of the verification required to ensure the specification is still complete, and all tests are passing. There's still work to specify and test more/all of the core/alloc/std library components, as well as third party crates, but from a toolchain perspective, it is much closer than you are giving them credit for. Unlike many proprietary C/C++ toolchains or verification suites, the majority of safety justification artifacts are publicly browseable here: https://public-docs.ferrocene.dev/main/index.html https://public-docs.ferrocene.dev/main/index.html. (I am a former founder of Ferrous Systems, and one of the people that pushed for the Ferrocene project to happen, but haven't worked there for a couple years and have no monetary stake in them anymore - I think they are just still doing the right thing, and doing it well.)
- lifthrasiir 2y ago
  Here the conformance refers to the Ferrocene language specification [1], which is being developed to be something akin to ISO/IEC 14882 for Rust in the future. In fact, this specification and the Ferrocene toolchain is meant to be ISO 26262 or IEC 61508 qualified, so it directly addresses your concern. The comment posted by jamesmunns will do much better job in describing the exact details than mine (both comments happened to be posted within 20 seconds, huh), so I'll just add that standards are meaningless by their own. Quite a lot of people criticizing Rust for the lack of formal standards seem to assume that such standards will immediately improve something, but that's never true. Standards are meaningful only when they are maintained and can be enforced; many historical standards failed to catch up and fell into the oblivion for this reason. It doesn't even matter much whether the specification is written in "formal" prose [2]. I feel they are missing the fundamental reason why language standards can be beneficial in principle. [1] https://spec.ferrocene.dev/ https://spec.ferrocene.dev/ [2] Which is never "formal" in the mathematical sense. (Mathematically formal language specifications are quite rare, examples include SML and WebAssembly.) In fact, such formal writing is slightly better than informal writing only because such writing also aims to be unambiguous, and the formality itself is not that important.

myworkinisgood 2y ago

C++ leadership are blinded by their own success and have no information of outside world.

bregma 2y ago

> analogous conformance test suites exist and are run automatically I'm not clear on how this is even possible. "Conformance" for C++ means adherence to the published ISO/IEC 14882 standard. In the case of Rust, it's "this is what the compiler does this week". Sure, a third party has built out a CI system for Rust and uses it provide evidence that Rust does what Rust does, but that's nothing similar to having evidence any random toolchain conforms to an accepted, published, international standard of what the C++ language does. When it comes to providing test evidence for functional safety (for example ISO 26262 or IEC 61508) I can point to Perennial or Sold Sands results and say with confidence that my toolchain does what it says on the C++ box. When it comes to Rust, I can point to Ferrous and say "the core Rust language when built using rustc does what the rustc developers claimed it should do last week, excluding any crates or libraries." To which claim are you going to trust the lives of you and your loved ones? They aren't analogous at all. There is just reams of marketing spin here and eventually it's going to kill someone.

jamesmunns 2y ago
This is entirely incorrect, Ferrocene manages a specification of a subset of the language here: https://public-docs.ferrocene.dev/main/specification/index.html https://public-docs.ferrocene.dev/main/specification/index.h..., including things like a traceability matrix that links the specification to the implementation and the verification: https://public-docs.ferrocene.dev/main/qualification/traceability-matrix.html https://public-docs.ferrocene.dev/main/qualification/traceab... For each release of Ferrocene, this is kept up to date, and the same as C or C++, what is specified and stable can be relied upon, and implementation details are implementation details, the same as it would be if you switched from LLVM w/ SolidSands' SuperTest suite to IAR or GreenHills' toolchains which may have varying impl details but still maintain conformance with the specification. The majority of safety critical teams will snapshot a single toolchain for the entire development lifecycle (sometimes updating if necessary, very rarely), but Ferrocene is releasing updates that are approaching the full Rust cadence (IIRC they've discussed going to every other release, so once every 12 weeks vs Rust's 6 week cadence), with all of the verification required to ensure the specification is still complete, and all tests are passing. There's still work to specify and test more/all of the core/alloc/std library components, as well as third party crates, but from a toolchain perspective, it is much closer than you are giving them credit for. Unlike many proprietary C/C++ toolchains or verification suites, the majority of safety justification artifacts are publicly browseable here: https://public-docs.ferrocene.dev/main/index.html https://public-docs.ferrocene.dev/main/index.html. (I am a former founder of Ferrous Systems, and one of the people that pushed for the Ferrocene project to happen, but haven't worked there for a couple years and have no monetary stake in them anymore - I think they are just still doing the right thing, and doing it well.)
lifthrasiir 2y ago
Here the conformance refers to the Ferrocene language specification [1], which is being developed to be something akin to ISO/IEC 14882 for Rust in the future. In fact, this specification and the Ferrocene toolchain is meant to be ISO 26262 or IEC 61508 qualified, so it directly addresses your concern. The comment posted by jamesmunns will do much better job in describing the exact details than mine (both comments happened to be posted within 20 seconds, huh), so I'll just add that standards are meaningless by their own. Quite a lot of people criticizing Rust for the lack of formal standards seem to assume that such standards will immediately improve something, but that's never true. Standards are meaningful only when they are maintained and can be enforced; many historical standards failed to catch up and fell into the oblivion for this reason. It doesn't even matter much whether the specification is written in "formal" prose [2]. I feel they are missing the fundamental reason why language standards can be beneficial in principle. [1] https://spec.ferrocene.dev/ https://spec.ferrocene.dev/ [2] Which is never "formal" in the mathematical sense. (Mathematically formal language specifications are quite rare, examples include SML and WebAssembly.) In fact, such formal writing is slightly better than informal writing only because such writing also aims to be unambiguous, and the formality itself is not that important.