5 ms·
> We saw this with Heartbleed too: given sure confidence that there is a vulnerability in a particular diff, skilled security researchers can find it very quick
by jared314 12y ago
> We saw this with Heartbleed too: given sure confidence that there is a vulnerability in a particular diff, skilled security researchers can find it very quickly.
That reminds me of the old QA technique / tactic of only telling the developers where a bug was found. Sometimes you don't even have to find an issue, you can just pick a complicated module.
- wglb 12y agoThere was a study done decades ago where code was seeded with a number of bugs. Developers were told that there were a number of bugs. They found that many, but the overlap was not total. That is to say, they found bugs that the study authors did not seed.
- xenophonf 12y agoIf you can dig up the citation, I'd love to read this. That's fascinating.
- wglb 12y agoI was not successful, but a sibling comment to yours https://news.ycombinator.com/item?id=8458030 https://news.ycombinator.com/item?id=8458030 provides a very useful description of the phenomenon.
- chetanahuja 12y agoPhew, I sure am glad that we instituted a company-wide policy of not putting any bugs in the code in the first place. Just makes life easier all around.
- wglb 12y agoWell, I remember at the time reading this that if one picked an arbitrary number, and told the developers that there were that many, they would find that many even if none were inserted.
- calinet6 12y agoYou laugh, but this is closer to most companies "quality policy" than otherwise. True quality comes from good process and systems, nothing more, nothing less.
- anonymfus 12y agoYou can use overlap to estimate total number of bugs: http://en.wikipedia.org/wiki/Lincoln_index http://en.wikipedia.org/wiki/Lincoln_index http://allendowney.blogspot.com/2014/07/last-year-my-occasional-correspondent.html http://allendowney.blogspot.com/2014/07/last-year-my-occasio...