6 ms·
There are plenty of studies at this point with empirical data showing that TDD is the way to go. The most famous is probably the Nagappan study at Microsoft and
by pmarreck 11y ago
There are plenty of studies at this point with empirical data showing that TDD is the way to go. The most famous is probably the Nagappan study at Microsoft and is 8 years old now: http://research.microsoft.com/en-us/groups/ese/nagappan_tdd.pdf http://research.microsoft.com/en-us/groups/ese/nagappan_tdd....
tl;dr If you consider a 90% reduction in bugs (and debugging, and unpredictable amounts of post-production debugging time, etc.) worth a 15-35% extra cost in upfront (but more predictable) development time... then you should be doing TDD, full stop.
If you can't figure out how to apply TDD to the problem, then look at the problem again. I/O is a common tripping point. Watch Gary Bernhardt's "Boundaries" talk for ideas: https://www.destroyallsoftware.com/talks/boundaries https://www.destroyallsoftware.com/talks/boundaries
I repeat... a 90% reduction in bugs. I think that's pretty damn huge. Read the paper, then google "TDD empirical data." Actually, I'll do it for you: https://www.google.com/search?q=TDD+empirical+data https://www.google.com/search?q=TDD+empirical+data
Trust me, I am all about doing things rationally and staying away from faddy things that have no data to back them up, but in this case, this one seems pretty solid.
Personal experience is that the code I write using TDD is better along almost all dimensions that factor into determining what "good code" even is: More modular, fewer dependencies, easier to maintain, easier to debug (when you must), smaller and more focused methods/functions, better modules/classes... These more intangible perks literally fall out of the sky like manna if you code with TDD.
It's admittedly tough to change to this habit, but now that I've been doing it, I would not go back. At all.
- sago 11y agoPersonal experience is that automated testing improved my productivity, TDD diminished it. But like any religious war, the criticism always is "you didn't do it right, if it didn't work for you." Thanks for the patronising 'let me google that for you'. I'm glad you saw the light, I'm content to remain a pagan. > If you consider a 90% reduction in bugs worth a 15-35% extra cost. I don't spend more than 31% of my time debugging, so 35% for a 90% reduction is nowhere near useful. I rarely spend 13% of the time on any bit of code debugging. And when I do I write unit tests as a debugging strategy, so it wouldn't have saved anything to write them beforehand. If you find yourself debugging for 30+% of your time, on new code you write, unless you write tests beforehand, then I respectfully suggest there are more pressing things to worry about in your practice.
- pmarreck 11y agoI wasn't trying to patronize, I was making it a few seconds more convenient for you in this case (this was NOT a literal "LMGTFY" back-hand!). And I wasn't trying to proselytize. And I know that "what works for me" might not be "what works for you." I just know that individually, and especially on any programming team, this has been a practice that has paid off in spades AND "feels better" as well. YMMV, I guess. I mean, there's a rational explanation for it: Your code produces states. What is a bug? An unexpected state. As long as you can contain/model ALL of those states, in your head, at all times, while working on the code, you can potentially write 100% bug-free code, at all times. But as soon as you cannot, then the tests will help contain the many possible states your code produces, and that your brain can no longer contain. And unless you are infinitely genius, your code will eventually reach a point where you simply cannot model all of its potential states in your head (this goes manyfold on a programming team where everyone is sharing bits of the "mass state" the code can produce.) Speaking of controlling unpredictable states and therefore bugs, FP often comes into the conversation as well, here's John Carmack on that topic: http://gamasutra.com/view/news/169296/Indepth_Functional_programming_in_C.php http://gamasutra.com/view/news/169296/Indepth_Functional_pro...
- sago 11y ago> the tests will help In my experience, proponents of TDD often end up actually defending testing when pushed, rather than TDD. As if TDD or write-and-forget code are the alternatives. > unless you are infinitely genius We've established, based on your own numbers, that you only need to spend less than 10-30% of your time debugging your code before it isn't worth it. There's no need to exclude the middle and pretend it requires infinite genius.
- pmarreck 11y agoI have alternated between TDD and post-testing. I've noticed that TDD forces me to think about the code in a better way, before actually coding it, than just going ahead and coding it and then figuring out how to test it after. This is by no means an easy practice to adopt, btw, especially after years of not doing so. I actually think TDD should be taught in schools and never even considered a separate aspect of coding; TDD should be integral to coding, period. If it wasn't discovered separately in the first place, it would just be called "coding." > that you only need to spend less than 10-30% of your time debugging your code before it isn't worth it That is a good point. It's worth it down the line. You're not factoring in future costs. You're not just reducing TODAY'S bugs by 90% with that increase in overall coding time, you're also vastly reducing the technical debt of the code in the future. You're also writing a permanent, provable spec for the code. What happens 5 years after you write your untested but debugged code and have to go back to it to add or fix something? How in the hell will you remember the whole mental model and know if you are in danger of breaking something? The answer is, you will not. And you will tread more useless water (time and effort) debugging the bugfixes or feature-adds or refactorings. Speaking of refactorings, they are almost impossible to do without huge risk unless you have well-written unit tests against the interface to the code being refactored. In short, if you do not write tests, you are literally placing a bet against the future of your code. Do you really want to do that? Do you consider your work THAT "throwaway"? That said, tests are no panacea... and I am NOT trying to sell them as one (which would be wrong). You might assert the wrong things, or miss testing negative cases (a common one is not testing that the right runtime error is thrown when inputs to the code do not conform). There are cases on record of well-tested code that has passed all tests (like satellite code) and still fails in the real world because of human error (both the code AND the test were wrong).
- Derbasti 11y agoHow do you TDD experimental/evolving code? I work on scientific algorithms, where there are typically only few lines of code that constantly evolve to improve some performance metric. I have been struggling for a while to integrate testing into my work. TDD is easy and perfectly suited for "normal" software development, where there is some kind of plan, and you are writing a lot of code that changes little after it is written. I'd be very interested in pointers on how to apply TDD to little code that changes a lot after it is initially written.
- pmarreck 11y agoDo you mean you work with evolutionary algorithms? One thing testing does require is determinism. In other words, given all input states X(1) through X(N), output Z should result 100% of the time. If that is not the case, then you haven't accounted for all input states (for example, code that looks at the time and acts based on that- the time is an oft-unaccounted-for input) or you have code that calls something like rand() without a seed. If you can get your code into a deterministic state, then it is testable.
- blub 11y agoThere are multiple studies (ref. Making Software, Oram & Wilson), the results are inconclusive. It's especially not clear whether it's worth it to do TDD in comparison with simple automated testing and other testing strategies. You are preaching by the way.
- pmarreck 11y ago> Making Software, Oram & Wilson Going to quote one of the authors (Hakan Erdogmus) from here: https://computinged.wordpress.com/2010/11/01/making-software-tdd-doesnt-work-architecting-is-mixed-scripting-is-great/ https://computinged.wordpress.com/2010/11/01/making-software... "I am one of the authors of the TDD chapter. Ours was a review of existing some 30+ studies of TDD. Yes, Nachi’s experiments (the MSR study mentioned above) were included. BTW, I wouldn’t have concluded that “tdd doesn’t work” based on our data. Rather, I would have conservatively concluded: there is moderate evidence supporting TDD’s external quality and test-support benefits, but there is no consistent evidence in one direction or other regarding its productivity effects." The only thing inconclusive was the increased productivity effect, not the code quality effect. > It's especially not clear whether it's worth it to do TDD in comparison with simple automated testing and other testing strategies. https://www.researchgate.net/publication/3188484_On_the_effectiveness_of_the_test-first_approach_to_programming https://www.researchgate.net/publication/3188484_On_the_effe... I've done every testing strategy. Test-none, BDD, TDD, test-after, integration testing, unit testing, GUI testing, you name it. If your code will last longer than a couple years, it is worth it to do extensive unit testing via TDD, integration testing of major/important workflows, a visual once-over by a QA person to make sure something didn't break the CSS, and that's it. If you are the founder of a startup and expect to sell within 2 years (and are thus incentivized not to go the TDD route), you better be correct in your bet or there will be technical-debt hell to pay. > preaching Espousing something that I have found over and over again in my programming career (testing since about 2004, TDD since about 5 years ago) is now "preaching"? Call me a prophet, then. I know exactly what I see. Don't listen to me though, I've only been coding since I was 8 in 1980... I encourage you to find out for yourself.
- blub 11y ago