3 ms·

So I got downvoted last time I brought this up, but if a large corporation hasn't fuzzed their products / code, doesn't this start to border on negligence? Bug

So I got downvoted last time I brought this up, but if a large corporation hasn't fuzzed their products / code, doesn't this start to border on negligence? Bugs will of course happen, but failure to fuzz products from companies that employ tens of thousands of people seems inexcusable.

oconnor663 8y ago
Fuzzing is great and undefined behavior is terrible, so I mostly agree with you. But negligence is a high bar, so let me add at least a small counterpoint: These big, old systems are full of crashes. Many of them have been triggered and investigated before, and determined not to be a security vulnerability, and so left as they are. If you fuzz these systems, you might find a lot more false positives than real exploitable vulnerabilities. It might be even worse than that, with the false positives being so many that they get in the way of the fuzzer actually discovering new issues. If these were newer projects, it might make sense to really clean the codebase up and fix every crash. That would make debugging tools more useful, and it would avoid future cases where some compiler update turns your "harmless" crash into an exploit. But in these legacy projects, that sort of cleanup would be very expensive, and many of the devs with the expertise to do a proper job of it have long since moved on. It really might not make economic sense to invest that much in cleanup, even if Microsoft has fully internalized the cost of vulnerabilities to their users.
da_chicken 8y ago
It should, but it doesn't. People haven't quite figured out that when software becomes infrastructure that software engineering should look a lot more like civil engineering.
- pjmlp 8y ago
  There are a couple of countries where signing off projects as software engineer does indeed require similar validation. Sadly we are still quite far from the ideal way. For that we need more escalations on company lawsuits so that it actually becomes more professional.
Someone1234 8y ago
After you've tried setting up and conducting fuzzing (rather than just reading someone else's success stories) you'll see why we cannot assume they haven't. Fuzzing isn't a binary state, you can spend weeks Fuzzing something, and then someone else with a different methodology can find dozens of new bugs. Fuzzing scripting languages is particularly complicated because the languages themselves have infinite state. The more complicated the Fuzzer the more coverage it has, but even writing a bespoke language specific Fuzzer is about as complicated as creating the language itself.

oconnor663 8y ago

Fuzzing is great and undefined behavior is terrible, so I mostly agree with you. But negligence is a high bar, so let me add at least a small counterpoint: These big, old systems are full of crashes. Many of them have been triggered and investigated before, and determined not to be a security vulnerability, and so left as they are. If you fuzz these systems, you might find a lot more false positives than real exploitable vulnerabilities. It might be even worse than that, with the false positives being so many that they get in the way of the fuzzer actually discovering new issues. If these were newer projects, it might make sense to really clean the codebase up and fix every crash. That would make debugging tools more useful, and it would avoid future cases where some compiler update turns your "harmless" crash into an exploit. But in these legacy projects, that sort of cleanup would be very expensive, and many of the devs with the expertise to do a proper job of it have long since moved on. It really might not make economic sense to invest that much in cleanup, even if Microsoft has fully internalized the cost of vulnerabilities to their users.

da_chicken 8y ago

It should, but it doesn't. People haven't quite figured out that when software becomes infrastructure that software engineering should look a lot more like civil engineering.

pjmlp 8y ago
There are a couple of countries where signing off projects as software engineer does indeed require similar validation. Sadly we are still quite far from the ideal way. For that we need more escalations on company lawsuits so that it actually becomes more professional.

Someone1234 8y ago

After you've tried setting up and conducting fuzzing (rather than just reading someone else's success stories) you'll see why we cannot assume they haven't. Fuzzing isn't a binary state, you can spend weeks Fuzzing something, and then someone else with a different methodology can find dozens of new bugs. Fuzzing scripting languages is particularly complicated because the languages themselves have infinite state. The more complicated the Fuzzer the more coverage it has, but even writing a bespoke language specific Fuzzer is about as complicated as creating the language itself.