5 ms·

Cool experiment! I found it amusing (and unsurprising) that the distribution of reported bugs matches what you would find in code reviews within most companies

Cool experiment! I found it amusing (and unsurprising) that the distribution of reported bugs matches what you would find in code reviews within most companies: a small handful of genuine issues buried in a giant pile of superficial nitpicks. Before anyone interjects, no, I'm not saying misspellings and stylistic inconsistencies don't matter. They matter to me a great deal--I am crazily OCD about that shit. Seeing 'shear' misspelled as 'sheer' throughout most of Unreal Engine 2 and 3 without being able to fix it (there were too many expected breaking changes to licensee code for such a cosmetic change to make sense) annoyed me like you wouldn't believe. But when I force myself to take a deep breath and consider the matter more rationally, it's clear that such issues are far down the list in importance. While I'm reviewing someone else's code and my OCD-ish code aesthetics sense is stroked, I always try to remind myself of that.

cperciva 15y ago
Yes, this is roughly the distribution I expected. Maybe a few more spelling mistakes than I expected, but not as many style bugs, so the "harmful bug" / "harmless bug" / "cosmetic bug" distribution matched reasonably closely. But I wouldn't call the harmless bugs "superficial nitpicks". Most of them could become problems later -- if other code changes result in the functions they are in being called with different parameters; or if an error is no longer treated as fatal but instead Tarsnap continues and retries the failed operation; or with more aggressive compiler optimizations. Better to fix the bugs before they cause problems than to wait for them to cause problems.
- psykotic 15y ago
  Oh, I meant the cosmetic bugs specifically. And to be clear, I'm not saying they shouldn't have been reported. Issues like that _should_ be fixed. I just found it all vaguely amusing for some reason because unless I keep myself in check I'm so much that grammar and spelling and formatting guy when it comes to code reviews.
  - cperciva 15y ago
    Ah, ok. My original motivation for offering bounties for those was as a proof-of-work for code reading -- I figured there would be enough of those that anyone who spent a while looking at the code would pick up a few, even if they couldn't find any "real" bugs. The fact that it satisfied my OCD is just a side benefit. ;-)
  - eyah 15y ago
    Well, the funny thing about cosmetic bugs is that they tend to cluster with other, more subtle bugs (in my experience, at least, I'm more likely to find substantive errors by concentrating on sections of code near the obvious cosmetic bugs).
jacques_chester 15y ago
You might find Watts Humphrey's book A Discipline for Software Engineering to be an interesting read. In it he describes a scheme where you track all your mistakes, no matter how tiny, and patiently develop ways to prevent them from happening in future. edit: Look, I know that Humphreys is the agile bogeyman, but why the downvote? Engage, don't merely click an arrow.
jrockway 15y ago
http://en.wikipedia.org/wiki/HTTP_referrer http://en.wikipedia.org/wiki/HTTP_referrer The misspelling referer originated in the original proposal by computer scientist Phillip Hallam-Baker to incorporate the field into the HTTP specification.[1] The misspelling was set in stone by the time of its incorporation into the standards document Request for Comments (RFC) 1945; document co-author Roy Fielding has remarked that neither "referrer" nor the misspelling "referer" were recognized by the standard Unix spell checker of the period.

cperciva 15y ago

Yes, this is roughly the distribution I expected. Maybe a few more spelling mistakes than I expected, but not as many style bugs, so the "harmful bug" / "harmless bug" / "cosmetic bug" distribution matched reasonably closely. But I wouldn't call the harmless bugs "superficial nitpicks". Most of them could become problems later -- if other code changes result in the functions they are in being called with different parameters; or if an error is no longer treated as fatal but instead Tarsnap continues and retries the failed operation; or with more aggressive compiler optimizations. Better to fix the bugs before they cause problems than to wait for them to cause problems.

psykotic 15y ago
Oh, I meant the cosmetic bugs specifically. And to be clear, I'm not saying they shouldn't have been reported. Issues like that _should_ be fixed. I just found it all vaguely amusing for some reason because unless I keep myself in check I'm so much that grammar and spelling and formatting guy when it comes to code reviews.
- cperciva 15y ago
  Ah, ok. My original motivation for offering bounties for those was as a proof-of-work for code reading -- I figured there would be enough of those that anyone who spent a while looking at the code would pick up a few, even if they couldn't find any "real" bugs. The fact that it satisfied my OCD is just a side benefit. ;-)
- eyah 15y ago
  Well, the funny thing about cosmetic bugs is that they tend to cluster with other, more subtle bugs (in my experience, at least, I'm more likely to find substantive errors by concentrating on sections of code near the obvious cosmetic bugs).

jacques_chester 15y ago

You might find Watts Humphrey's book A Discipline for Software Engineering to be an interesting read. In it he describes a scheme where you track all your mistakes, no matter how tiny, and patiently develop ways to prevent them from happening in future. edit: Look, I know that Humphreys is the agile bogeyman, but why the downvote? Engage, don't merely click an arrow.

jrockway 15y ago

http://en.wikipedia.org/wiki/HTTP_referrer http://en.wikipedia.org/wiki/HTTP_referrer The misspelling referer originated in the original proposal by computer scientist Phillip Hallam-Baker to incorporate the field into the HTTP specification.[1] The misspelling was set in stone by the time of its incorporation into the standards document Request for Comments (RFC) 1945; document co-author Roy Fielding has remarked that neither "referrer" nor the misspelling "referer" were recognized by the standard Unix spell checker of the period.