3 ms·

That's pretty paranoid if you're already trusting the source or binary. Have you previously ensured that same source acts non-maliciously on all malicious input

That's pretty paranoid if you're already trusting the source or binary. Have you previously ensured that same source acts non-maliciously on all malicious inputs and runs on secure OS? (Sources of most attacks.)

Gibbon1 7y ago
In medical devices you validate the binary. The problem is if you rebuild the code and now it's different then what? Consider that compilers sometimes produce the wrong code. I've hand compilers do this to me. Build, run tests. Test fails. Investigate, assembly looks 'weird'. Rebuild and now it's correct. GAH!
- nickpsecurity 7y ago
  That makes sense. It could help there depending on what validation requirements are. High-assurance systems is my thing btw. There's a few ways we approach this issue. One is a certifying compiler (eg CompCert C) that's verified and tested to basically never screw up. CompCert got SIL qualification recently with Solid Sands partnering to validate them, too. Another I like is equivalence testing across multiple binaries from multiple compilers with a large, automatically-generated test suit and fuzzers. Such test generators are already great for bug hunting. You just rerun them through the same app compiled with other compilers. Helps catch app and compiler bugs. If you have CompCert, I have another trick: re-compile with CompCert, re-test each case that failed, and any that suddenly pass were likely compiler bugs. The multiple compilers approach should catch them, though. Now, I should point out that what reproducible builds is addressing are often changes in hashes that aren't changes in functionality. Just stuff like timestamps or symbols. You should be able to show the regulators that the only thing that changed was that kind of stuff. Maybe we need tools that automate that, too, showing diffs annotated by what does or doesn't affect execution. Anyway, what exactly do they require of you for source-to-object correspondence? And is this U.S. F.D.A. or a non-U.S. regulator?
  - Ididntdothis 7y ago
    “Anyway, what exactly do they require of you for source-to-object correspondence?” In my case this is not really clear. You have quite a bit of leeway but you never know when you will get rejected. We had things that are approved a few years ago being rejected now because some rules have been tightened. With reproducible builds you could save a lot of time validating build chains. It would be a huge timesaver and probably also allow us to be more creative with the build chain because if the binary is the same you know it worked.

Gibbon1 7y ago

In medical devices you validate the binary. The problem is if you rebuild the code and now it's different then what? Consider that compilers sometimes produce the wrong code. I've hand compilers do this to me. Build, run tests. Test fails. Investigate, assembly looks 'weird'. Rebuild and now it's correct. GAH!

nickpsecurity 7y ago
That makes sense. It could help there depending on what validation requirements are. High-assurance systems is my thing btw. There's a few ways we approach this issue. One is a certifying compiler (eg CompCert C) that's verified and tested to basically never screw up. CompCert got SIL qualification recently with Solid Sands partnering to validate them, too. Another I like is equivalence testing across multiple binaries from multiple compilers with a large, automatically-generated test suit and fuzzers. Such test generators are already great for bug hunting. You just rerun them through the same app compiled with other compilers. Helps catch app and compiler bugs. If you have CompCert, I have another trick: re-compile with CompCert, re-test each case that failed, and any that suddenly pass were likely compiler bugs. The multiple compilers approach should catch them, though. Now, I should point out that what reproducible builds is addressing are often changes in hashes that aren't changes in functionality. Just stuff like timestamps or symbols. You should be able to show the regulators that the only thing that changed was that kind of stuff. Maybe we need tools that automate that, too, showing diffs annotated by what does or doesn't affect execution. Anyway, what exactly do they require of you for source-to-object correspondence? And is this U.S. F.D.A. or a non-U.S. regulator?
- Ididntdothis 7y ago
  “Anyway, what exactly do they require of you for source-to-object correspondence?” In my case this is not really clear. You have quite a bit of leeway but you never know when you will get rejected. We had things that are approved a few years ago being rejected now because some rules have been tightened. With reproducible builds you could save a lot of time validating build chains. It would be a huge timesaver and probably also allow us to be more creative with the build chain because if the binary is the same you know it worked.