4 ms·
What does $1265 of bugs look like?
- thaumaturgy 15y agoYou have an astounding dedication to your craft.
- cperciva 15y agoIt makes me sad that this is considered abnormal, never mind astounding.
- jgrahamc 15y agoI agree with that sentiment. I have a piece of C code that I license to companies (it does multi-label classification of documents---mostly email) and I have very good test coverage and have been obsessive about its design and operation. Since it does a lot of low level C tricks for speed I wanted to be absolutely sure that it wouldn't crash, and it has to work on Windows (32 and 64), Solaris, HP/UX, AIX, Linux, ... The bottom line is that being obsessed about the stability of the code has meant that in the five years it's been shipping not a single client has reported a bug and I have never heard of the code crashing. This actually turned out to be a problem for me, because I had a hard time getting some people to renew maintenance on the code because it never failed.
- TheAmazingIdiot 15y agoI deign to mention it, but if you were to add in a "bug" that would safely 'crash' at sparse but random-ish intervals, you could charge for maintenance. Maintenance would consist of lowering the multiplier of "crash" occurance. So your Program is always improving. I see little difference between this and the way hardware makers cripple lower end hardware You can sell maintenance then. Slimy, i know. But business is business.
- nawariata 15y agoUsing the car analogy, crippling hardware to make it slower is an equivalent of putting speed limiter on an engine (admittedly for different reasons) which still makes it fully operational. Purposefully introducing bugs is like making airbags go off at random times so you could charge for service. Nice business practice.
- jacques_chester 15y agoThis tactic might exist in the wild, based on some stories I've seen on DailyWTF, but I would be amazed if it didn't result in lawsuits. One of these days those "no liability" clauses are going to be removed by legislation or a blockbuster court case and whole sewerage plants are going to smash into fan factories.
- yuvipanda 15y agoJust curious - did the fact that it was hard to get people to renew maintenance contracts have any effect on your code quality at all?
- jgrahamc 15y agoNo, I take bugs personally. If you actually take bugs personally they hurt and you don't want to have to deal with them. Thus you make them go away.
- ColinWright 15y agoFor literally decades I have advocated this attitude. Time and time again my efforts are beaten back with "Bug-Free doesn't sell" and "Bug-Free doesn't pay" and, most insidious, "Bug-Free is impossible." This last is then followed by * You have to verify the compiler, * You have to verify the hardware, * You have to verify that there wasn't an error in loading the code, * You have to use radiation-hardened hardware * ... and so on. They miss the point! Bugs in your code are way more likely than any of the other problems, and they matter. I've generally given up, but at least it's nice to see the attitude here not being instantly shouted down.
- jgrahamc 15y agoI'm with you on "Bugs in your code are way more likely than any of the other problems, and they matter." This is very true. A while back Eric Raymond had a post called "When you see a heisenbug in C, suspect your compiler’s optimizer" (http://esr.ibiblio.org/?p=1705 http://esr.ibiblio.org/?p=1705) which I felt was very wrong and responded with "A bad workman blames his tools" (http://blog.jgc.org/2010/02/bad-workman-blames-his-tools.html http://blog.jgc.org/2010/02/bad-workman-blames-his-tools.htm...). As might have been predicted the bug that Raymond was talking about wasn't caused by the optimizer at all (http://esr.ibiblio.org/?p=1705#comment-248328 http://esr.ibiblio.org/?p=1705#comment-248328). It's depressing to think that when your code works, it's almost always your fault.
- 15y ago
- cperciva 15y agoThis is why I've never believed in the "charge for support" model for funding open source software -- it creates a financial incentive to have hard-to-use bug-ridden code. With Tarsnap, my incentives are perfectly aligned: Writing good code results in happy Tarsnap users, which results in more Tarsnap users.
- cookiecaper 15y agoWell, "charge for support" works for amalgamations of code like Red Hat, but I agree it's probably not ideal for individual projects (at least small projects). I prefer offering services related to the code -- not only bugfixes for cash, but also new features and installation. CodeWeavers develops WINE this way and I think they're doing pretty well.
- antifuchs 15y agoLet's say an admirable dedication to your craft, then. I'm definitely glad you put out these bounties - you set a very good example for us all.
- psykotic 15y agoCool experiment! I found it amusing (and unsurprising) that the distribution of reported bugs matches what you would find in code reviews within most companies: a small handful of genuine issues buried in a giant pile of superficial nitpicks. Before anyone interjects, no, I'm not saying misspellings and stylistic inconsistencies don't matter. They matter to me a great deal--I am crazily OCD about that shit. Seeing 'shear' misspelled as 'sheer' throughout most of Unreal Engine 2 and 3 without being able to fix it (there were too many expected breaking changes to licensee code for such a cosmetic change to make sense) annoyed me like you wouldn't believe. But when I force myself to take a deep breath and consider the matter more rationally, it's clear that such issues are far down the list in importance. While I'm reviewing someone else's code and my OCD-ish code aesthetics sense is stroked, I always try to remind myself of that.
- cperciva 15y agoYes, this is roughly the distribution I expected. Maybe a few more spelling mistakes than I expected, but not as many style bugs, so the "harmful bug" / "harmless bug" / "cosmetic bug" distribution matched reasonably closely. But I wouldn't call the harmless bugs "superficial nitpicks". Most of them could become problems later -- if other code changes result in the functions they are in being called with different parameters; or if an error is no longer treated as fatal but instead Tarsnap continues and retries the failed operation; or with more aggressive compiler optimizations. Better to fix the bugs before they cause problems than to wait for them to cause problems.
- psykotic 15y agoOh, I meant the cosmetic bugs specifically. And to be clear, I'm not saying they shouldn't have been reported. Issues like that _should_ be fixed. I just found it all vaguely amusing for some reason because unless I keep myself in check I'm so much that grammar and spelling and formatting guy when it comes to code reviews.
- cperciva 15y agoAh, ok. My original motivation for offering bounties for those was as a proof-of-work for code reading -- I figured there would be enough of those that anyone who spent a while looking at the code would pick up a few, even if they couldn't find any "real" bugs. The fact that it satisfied my OCD is just a side benefit. ;-)
- LeafStorm 15y agoI clicked on the link, and it displayed the "Aw, snap! Something went wrong while displaying this Web page." page. I suppose that $1265 of bugs could look like that.
- ralph 15y agoI've spent a while looking for tarsnap bugs; to date I count 86 email threads with Colin to report my findings. There are a couple of things I think are pretty good about the scheme Colin has devised. Unlike some other bounty schemes, there are small bounties available. One can look for ages without finding a $50 bounty, so long that boredom would probably set in and the hunt would be abandoned. But coming across the odd $1 or $5 bounty keeps interest going at small cost to Colin and still improves the overall source quality. It's almost like a bit of a game, a treasure hunt, where small pickings on route keep one's hopes up of rich pickings down the road. A game that can be set down and picked up later as time allows. Secondly, Colin was very fair in judging the bounty to be awarded. It soon became apparent that I could trust his judgement; whenever I thought a bug was undervalued another would come along that might be overvalued. On mentioning this to him, it turns out it was sometimes deliberate; he'd be wavering and thought it fair to even out which side of the boundary the bug fell. As for the OCD, I boggle a bit at some of those style bugs and don't think they were all me! :-) I do recall sending in some trivial thing suggesting it wasn't bountiful but perhaps should be fixed anyway; Colin generously still gave a dollar. I thought it worth reporting even without expecting a bounty because I think anything that interrupts the flow of the reader, causes them to pause, irked at a possessive "its" misspelt as "it's", detracts from the odds of them spotting something more serious. Overall, a well crafted and executed bounty scheme. In some ways I'm surprised other companies don't do something similar, even if it was with closed source under an NDA. If the feedback is prompt then the bounty hunter can decide to stop if the bounties aren't awarded fairly enough in his opinion, or the company can cancel, perhaps having shelled out their budget or unhappy with the quality of the bugs, without the hunter having wasted much time since the last bounty.
- olliesaunders 15y agoI hadn’t ever encountered this idea (bug bounties) before and I think it’s great. Only thing that bothers me is that, if you were to run the scheme for long enough it might start to become a victim of its own success: wary of the bug hunters your programmers would be so much more careful (think especially about the harmless and stylistic stuff) that there wouldn’t be any small things available to find any more and bug hunters would get bored and give up. To resolve this you could only conduct the bounties periodically so there’s always enough of a backlog of bugs to be found.
- mhlakhani 15y agoI don't know about the others, but I think the bug bounty is a good idea. I mostly went over the code so I could learn something about good C coding conventions. Plus I wanted to try out tarsnap without having to go through the hassle of getting access to a credit card; the bug bounty is pretty helpful for that use case.
- ph0rque 15y agoHmmm, cperciva mentions 211 bounties, but 53 + 70 + 60 + 11 = 194 bugs. What were the other 17 bugs; duplicate reports?
- cperciva 15y agoIt was about 70 style bugs, not exactly. A few bounties covered documentation, and I awarded one to someone who told me how to fix some CSS on the website.
- ralph 15y agoI came across a comment on the CSS in the HTML of the site offering a bounty for the fix, realised I didn't know the answer, and pointed it out to a LUG friend of mine who I thought would. He did and got the sole $5 bounty. :-)