3 ms·
Heck, the subtitle even says it: "It's probably still true, though, says formal methods expert" What's more interesting anyway is the circumstances around the
by nerdponx 5y ago
Heck, the subtitle even says it: "It's probably still true, though, says formal methods expert"
What's more interesting anyway is the circumstances around the nonexistence of the study. It's not a case of falsified research! It's a case of someone citing apocrypha as fact, and then other people citing that citation, etc. And nobody questioned it because it's so obviously true!
- whoisthemachine 5y agoTruly seems to get to the root of some of software's troubles...
- Arnt 5y agoI came across it in a book that we were supposed to read at university thirty years ago, but the book was outrageously expensive. It cost more than all of my other textbooks put together, and the university book shop didn't have it and told me I'd have to wait months. It's probably different now that I can click around a little on some web sites and three days later a book arrives from New Zealand. But that was thirty years ago. Can't remember how I read that book, but I did and I know that most others in the course didn't. I find it very easy to believe that people might assume things about what it said rather than actually read the text, and hand it down to others as fact. And so what's illustrative anecdote transforms into assumedly well-researched fact.
- Arnt 5y agoReplying to myself... I wonder what book that was. It was a large slim hardback published sometime in the 1970s, around A4, so much earlier than the 1987 book. I don't have any course materials any more. As I recall, they argued that making a change in the requirements specification was ~10 times cheaper than finding and fixing a bug while writing the implementation, which in turn was ~10 times cheaper than fixing it after delivery, ie. on other people's production computers at other site. There was some argument around it, but the text didn't imply exactness, as far as I can remember.
- AndrewDucker 5y agoIs it obviously true though? More expensive, sure? But 100x? Is it not 50x? Or 300x? Without some actual measurements I'd want to put some very very big error bars on my feelings there!
- laurent92 5y agoExactly: If bugs cost 50x more in production, but only bugs that are relevant for the core product are reported , then I’ve won 90% of the development time. That’s what I did for my startup (no tests, no 3-tier), and it was extremely valuable seeing all reports focus on a few areas, half of the reports being requests for improvement, extremely useful in determining 90% of the critical path and proving the ballpark of revenue that people would put into it, and since we can now hire, we’re now fully 3-tier with modern tech an all. Good management is judging where to put the team’s energy.
- wongarsu 5y agoI would be very surprised if there is a meaningful number without adding much more context. Some early stage startup might be able to fix a low impact bug in production at only 10x cost, but I wouldn't be surprised if the two bugs exhibited in the first test flight of goings Boing's Starliner crew capsule have cost them over a million times as much as fixing them earlier, with the retests, the massive timeline slip, and all the knock on effects from the delay that majorly benefited their most promising competitor
- throwawayboise 5y agoAnd it all depends on how you define "bug." Misallocating memory? Use after free? Off-by-one? Probably cheap to fix even in production. A misunderstood or missed requirement? Sure, that could be very expensive if it affects a data structure or some other interface with other systems that then also have to change. But that goes beyond what I normally think of as a "bug".
- AndrewDucker 5y agoThat depends on how expensive it is to get your code into production, how many other dependencies need to move in lockstep, how much manual regression testing there is, etc. Fixing a ten-second bug on my PC might easily be 100x cheaper than organising a release.
- taeric 5y agoThat subtitle really just reads as a doubling down on information that is in dispute. That is, we have research that shows you strengthen beliefs by weakening evidence. Now, to your point. This weakened the evidence, it did not falsify it. Does feel very similar.