5 ms·

If you find this many critical bugs, there is no doubt that there are more, since it is indicative of the overall code quality. Still, what they're doing is goo

by Kenji 10y ago

If you find this many critical bugs, there is no doubt that there are more, since it is indicative of the overall code quality. Still, what they're doing is good work and crypto remains hard. I am glad they're doing it.

AdmiralAsshat 10y ago
That reasoning strikes me as flawed. It would be like submitting a term paper to a teacher after he proofread your draft, with all of his suggested fixes, yet his final response is "Oh sure, you fixed all the problems I found. But I found so many before, what about the ones I didn't find? C+" This was a third-party audit, and VeraCrypt fixed all of the critical ones that they found. All software has bugs. To find some that are critical is not necessarily indicative of the quality of code on the whole.
- water42 10y ago
  actually, it would be more like students submitting 100,000 term papers to a teacher after the teacher proofread the drafts and the students made all his suggested fixes. auditing a large codebase certainly isn't as easy as proofreading a paper. it's not a "one and done" scenario. especially when you also factor in the security requirements of encryption software. the world won't end if there's a missed spelling mistake or two in a term paper, but if there are critical vulns in VeraCrypt the entire software could be rendered useless
- kkl 10y ago
  I do not think your analogy applies here. Finding 8 critical-risk bugs in a project from an audit is bad news. It is highly suggestive that the codebase is riddled with flaws. Smoke, fire, yadda yadda.
dimino 10y ago
If there were more bugs, why weren't they found? This is a form of "slippery slope" argument that holds no weight as to the security of VeraCrypt, one way or another.
- greenshackle2 10y ago
  It's not slippery slope. The slippery slope fallacy is arguments of the form 'if we allow x, y and z will necessarily occur, therefore we can't allow x'. (Note that the slippery slope fallacy is not a pure logical fallacy, the argument is valid if the slope is, in fact, slippery). Between two pieces of software with the same function, if the same audit process finds 10 critical bugs in software A, and 100 critical bugs in software B, which do you think is more likely to have undiscovered bugs? I'd say B, since no bug discovery process is perfect. For example, assuming a ~90% hit rate, you'd expect there is ~1 undiscovered bug in A, and ~10 in B. (Note that I am not saying VeraCrypt is bad or that you shouldn't use it. I have no idea how it compares to similar software. All I'm arguing is that, ceteris paribus, discovering more critical bugs is at least weak evidence that more undiscovered critical bugs exist, since no bug-discovery process is perfect.)
  - dimino 10y ago
    "If we found bug x, y and z will necessarily be found, therefore VeraCrypt is insecure." That's the argument being made implicitly, and the argument you're reiterating. Saying, "Who knows if there are more bugs?" is a useless statement, you simply can't do anything with it. That's the nature of bugs, if you can't identify or find them, you can't claim they exist. 100 bugs in software B does not imply the 101st bug, is my point. There's no such thing as a bug "hit rate".
    - greenshackle2 10y ago
      >If we found bug x, y and z will necessarily be found, therefore VeraCrypt is insecure. I did not make that argument, I made a probabilistic argument. Trying to apply deductive logic fallacies to probabilistic arguments is a type error. >if you can't identify or find them, you can't claim they exist. I made no absolute claim about the existence or nonexistence of bugs, only about the expected number of bugs, that is, a probabilistic estimate. > There's no such thing as a bug "hit rate". Of course there is, it is, trivially, the number of bugs found by the audit / the total number of bugs. Look at this way: I have two auditors, Alice and Charles. Charles is known to be pretty sloppy, he looks over code quickly, skip parts, etc, but the bugs he finds are usually actual bugs. Alice is extremely rigorous and misses very few bugs. I have 2 pieces of software, SuperCrypt and UltraCrypt. Charles audits SuperCrypt and finds 137 critical bugs. He audits UltraCrypt and finds only one. I know Alice will review both code bases next week. Where do you think she'll find more bugs?
wojcech 10y ago
To add onto the admirals statement, I have two kinds of critical bugs mentally: Critically dangerous, but understandable(complex crypto/ interactions). Those do not indicate low code quality. Critically stupid/unwise(you know what these are, e.g. the libressl rant list). These do.

by Kenji 10y ago

AdmiralAsshat 10y ago

That reasoning strikes me as flawed. It would be like submitting a term paper to a teacher after he proofread your draft, with all of his suggested fixes, yet his final response is "Oh sure, you fixed all the problems I found. But I found so many before, what about the ones I didn't find? C+" This was a third-party audit, and VeraCrypt fixed all of the critical ones that they found. All software has bugs. To find some that are critical is not necessarily indicative of the quality of code on the whole.

water42 10y ago
actually, it would be more like students submitting 100,000 term papers to a teacher after the teacher proofread the drafts and the students made all his suggested fixes. auditing a large codebase certainly isn't as easy as proofreading a paper. it's not a "one and done" scenario. especially when you also factor in the security requirements of encryption software. the world won't end if there's a missed spelling mistake or two in a term paper, but if there are critical vulns in VeraCrypt the entire software could be rendered useless
kkl 10y ago
I do not think your analogy applies here. Finding 8 critical-risk bugs in a project from an audit is bad news. It is highly suggestive that the codebase is riddled with flaws. Smoke, fire, yadda yadda.

dimino 10y ago

If there were more bugs, why weren't they found? This is a form of "slippery slope" argument that holds no weight as to the security of VeraCrypt, one way or another.

greenshackle2 10y ago
It's not slippery slope. The slippery slope fallacy is arguments of the form 'if we allow x, y and z will necessarily occur, therefore we can't allow x'. (Note that the slippery slope fallacy is not a pure logical fallacy, the argument is valid if the slope is, in fact, slippery). Between two pieces of software with the same function, if the same audit process finds 10 critical bugs in software A, and 100 critical bugs in software B, which do you think is more likely to have undiscovered bugs? I'd say B, since no bug discovery process is perfect. For example, assuming a ~90% hit rate, you'd expect there is ~1 undiscovered bug in A, and ~10 in B. (Note that I am not saying VeraCrypt is bad or that you shouldn't use it. I have no idea how it compares to similar software. All I'm arguing is that, ceteris paribus, discovering more critical bugs is at least weak evidence that more undiscovered critical bugs exist, since no bug-discovery process is perfect.)
- dimino 10y ago
  "If we found bug x, y and z will necessarily be found, therefore VeraCrypt is insecure." That's the argument being made implicitly, and the argument you're reiterating. Saying, "Who knows if there are more bugs?" is a useless statement, you simply can't do anything with it. That's the nature of bugs, if you can't identify or find them, you can't claim they exist. 100 bugs in software B does not imply the 101st bug, is my point. There's no such thing as a bug "hit rate".
  - greenshackle2 10y ago
    >If we found bug x, y and z will necessarily be found, therefore VeraCrypt is insecure. I did not make that argument, I made a probabilistic argument. Trying to apply deductive logic fallacies to probabilistic arguments is a type error. >if you can't identify or find them, you can't claim they exist. I made no absolute claim about the existence or nonexistence of bugs, only about the expected number of bugs, that is, a probabilistic estimate. > There's no such thing as a bug "hit rate". Of course there is, it is, trivially, the number of bugs found by the audit / the total number of bugs. Look at this way: I have two auditors, Alice and Charles. Charles is known to be pretty sloppy, he looks over code quickly, skip parts, etc, but the bugs he finds are usually actual bugs. Alice is extremely rigorous and misses very few bugs. I have 2 pieces of software, SuperCrypt and UltraCrypt. Charles audits SuperCrypt and finds 137 critical bugs. He audits UltraCrypt and finds only one. I know Alice will review both code bases next week. Where do you think she'll find more bugs?

wojcech 10y ago

To add onto the admirals statement, I have two kinds of critical bugs mentally: Critically dangerous, but understandable(complex crypto/ interactions). Those do not indicate low code quality. Critically stupid/unwise(you know what these are, e.g. the libressl rant list). These do.