3 ms·

I didn't like the tone of the post either. I do think that this needs to be settled, though. If we're ignoring a practice that guarantees an improvement in the

• mhd 16y ago
  That study you cite has a very lenient definition of "comparable team". Not too surprising, most methodology studies that I've seen can't exactly claim scientific rigor. (Yes, it's hard to do when it comes to real-world projects, but other fields of scientific study manage better.)
  • lyudmil 16y ago
    "Lenient" would imply that their approach skewed the results. If that's what you meant, could you explain why you say that? I'm hoping your explanation could lead to a testable hypothesis. If you meant that they weren't rigorous in controlling variables related to team composition, I'd agree. However, I think those variables can safely be ignored if the study is repeated and validated many more times (which the authors acknowledge is a work in progress).
    • mhd 16y ago
      Project A: TDD Team: 6 people, 24 man-months, no legacy code, 6 KLOC; Trad Team: 2 people, 12 man-months, no indication whether it was a legacy project, whether they used unit testing at all (and if yes, what their coverage was). Project B: TDD Team: 8 people, 46 man-months, no legacy code, 26 KLOC Trad Team: 12 people, 144 man-months, 149 KLOC, again no indication whether it was legacy, used unit testing etc. The report mentions that they were comparable, because both teams reported to the same manager (Not too surprising for Microsoft). I hope that there's more info about this study and that I'm a bit overreacting, but just given these tables, I wouldn't agree that we're talking about comparable teams and projects here. So does this test unit-testing? Does it test TDD? Does it test development without legacy code vs legacy code? And even if you would get results, would it be the same for people with different experience levels? (Which is a problem for academic studies)
• gte910h 16y ago
  >If we're ignoring a practice that guarantees an improvement in the quality of our code and the speed of our delivery without a very good reason, then we are being unprofessional. Very good reason? Yes, it's very costly and only catches a certain sort of error. Would clients prefer 25% more features or the set you have with a full suite of unit tests? Many would prefer the 25% more features. Additionally, with that same amount of money you could spend doing TDD, you can do any number of OTHER practices which ALSO increase code quality, including, formal design reviews, informal design reviews, informal code reviews, formal code reviews, desk checking of code, automated analyzers, running through profilers, running through memory leak detectors, modeling, prototyping, component testing, integration testing, regression testing, system testing, beta testing and many more techniques. All of these take TIME and MONEY. TDD is just one of the many things you can do if your client/employer cares about code quality, and not necessarily the best thing. I mean formal inspections (according to code complete 2, pg 470) and formal design reviews are always much more efficacious at finding bugs than unit testing (which is what TDD revolves around). According to that, we shouldn't be spending the quality improvement time doing TDD based procedures then, but instead pouring over the code in review sessions. Secondly, many other techniques CAN yield better results than TDD, and in less time (but don't always). These include protoyping, personal desk checking, system tests, and beta tests with more than 1000 users. So many adherents of methodology X don't accurately scope in the tradeoffs of their costs, etc, when advocating their system. Morning scrumm meetings sacrifice people who want to have long (and therefore, more unpredictable commutes). People who like automated function comment headers don't count how much time people spend filling in those header boxes. People who like variable naming conventions don't spend enough time understanding what that does to the people who have to constantly change the way they name them. People who like full waterfall don't understand the morale crushing cost of huge changes after large design/documentation pushes like that. I could go on and on. TDD has costs. Those costs aren't necessarily worth it. Sometimes they are, sometimes they're not. The people who say "it saves you time" are full of crap. It may, sometimes, save some people, some time. All the tests, if they're at all worthwhile, both take time to write and maintain. I've been on projects which use it and don't, and while there was a lower defect rate on certain types of bugs, those weren't always very important, nor was the time cost of writing the tests worth it, as those projects had several high value features which couldn't be completed due to time.