4 ms·

Developers, confess why you don't TDD

3 points by lancerkind 9y ago

andreasgonewild 9y ago
I'm sure the world learned a thing or two from the agile movement. But like all movements it got twisted into semi-religious caricatures of the original message, like this post. Event Kent Beck admits skipping tests when he feels like it, no big deal. I hereby confess; I don't give a shit about icons, worship and obedience; I am creating.
- lancerkind 9y ago
  How was TDD presented/introduced to you?
  - andreasgonewild 9y ago
    I introduced myself to TDD/eXtreme programming long before anyone was talking about it, read all the books and preached the ideas. Since then I've had the misfortune of seeing several fucked up interpretations in the wild, from hip-shooting startups to ISO-certified Scrum shops; and they all manage to miss the entire point of the exercise; which is applying common sense; distributing authority and focusing on the important parts, not least creativity. Agile is kind of like the Jet Kune Do or Ving Tsun of software development; since it contains almost no form in itself; it's very easy embrace and extend into whatever you fancy, and therefore very easy to corrupt.
    - lancerkind 9y ago
      I'm curious how much time was spent trying to write test driven code before deciding it wasn't working for you. Did you succesfully write a few small programs using TDD (coding katas)? Or did you apply TDD to your project at work? For how many hours or lines of code did you try TDD before deciding it wasn't going to work?
eesmith 9y ago
TDD doesn't really seem to be useful, at least, not as clearly useful as its proponents generally advocate. That is, it's generally advocated as a clear win, but of the various attempts to quantify it, it's a mix at best. For example, https://www.computer.org/csdl/trans/ts/2013/06/tts2013060835.html https://www.computer.org/csdl/trans/ts/2013/06/tts2013060835... says "This paper provides a systematic meta-analysis of 27 studies that investigate the impact of Test-Driven Development (TDD) on external code quality and productivity. The results indicate that, in general, TDD has a small positive effect on quality but little to no discernible effect on productivity." It then goes on about subgroup analysis, but we must take that with a grain of salt as subgroup analysis is more likely to have "statistically significant" correlations due to random fluctuations. When I try TDD on my own projects, I find that I lock in the internal API too quickly, that is, I test against internal functions, because they are easy to test. Then when I refactor, I find I avoid refactoring which might, for example, merge a couple of functions into one or otherwise change that internal API, because of the effort of rewriting the tests to meet the new API. I work best with "Spike and Stabilize", that is, a spike solution, flesh it out, then once the API is mostly finished, develop the tests. I also use coverage testing to help identify which tests are missing. Side note: I hate it when TDD people say that TDD code naturally has 100% coverage. It doesn't. At a process level, the refactor can include "replace algorithm", but the new algorithm may have different edge cases than the original algorithm, so require additional tests to ensure correctness. For example, replace a simple quicksort with timsort, because it's possible to build a test case which causes quicksort to hit quadratic worst-case performance. But if the input has 63 elements or fewer then only the insertion sort part of timsort will be tested. You need a very different test suite to really exercise timsort, compared to what's needed to test a simple quicksort. Or for a simpler example, performance analysis may say that the case n=1 takes 90% of the time, so rather than use the slower general-purpose algorithm, the new strategy is to special-case n=1. However, deeper in the code there was already special support for n=1, which is now no longer used. There's no way TDD can identify this dead code. Has there been any analysis of coverage analysis on TDD-based projects, where coverage was never evaluated before, to see what the coverage actually is?
- lancerkind 9y ago
  Regarding coverage comment, your right in that code can contain capabilities (or bugs) beyond the TDD coded conceived. (If they weren't conceived then it could be a bug or things that don't happen in practical operations.) The TDD people are right in that if you follow the TDD process, all your code comes up as covered by a code coverage tool. But this is only coverage. Not all possible functionality coverage. I imagine "all possible functionality coverage" is an intractable problem in the same class as the Halting Problem. Branch/line coverage should be near 100% for TDD code. (Some I/O coupling areas are intentionally saved for other types testing as they aren't unit testable.) It's an easy thing to measure. It's a common industry metric. It's not the whole picture but it is something. Measuring code coverage on TDD created software that was built without monitoring the code coverage, then later use code coverage at the end as an evaluation: I've only a few anecdotal (two) instances where I did this and I found the code coverage to be well above 95% for the code base and at 100% for modules that were pure logic. In one of those instances, after many rounds of defect injection, I discovered one or two locations where another unit test could be added. All in all, I've been pretty happy with TDD. But who cares? I want to hear from the industry why others aren't doing it. :-) Thank you eesmith, I enjoyed the discussion! How about hearing some more from others? What's stopping you from doing TDD? Is it because you don't believe it? If you don't believe it, how was it presented to you? Did you try it?

3 points by lancerkind 9y ago

I'm sure the world learned a thing or two from the agile movement. But like all movements it got twisted into semi-religious caricatures of the original message, like this post. Event Kent Beck admits skipping tests when he feels like it, no big deal. I hereby confess; I don't give a shit about icons, worship and obedience; I am creating.

lancerkind 9y ago
How was TDD presented/introduced to you?
- andreasgonewild 9y ago
  I introduced myself to TDD/eXtreme programming long before anyone was talking about it, read all the books and preached the ideas. Since then I've had the misfortune of seeing several fucked up interpretations in the wild, from hip-shooting startups to ISO-certified Scrum shops; and they all manage to miss the entire point of the exercise; which is applying common sense; distributing authority and focusing on the important parts, not least creativity. Agile is kind of like the Jet Kune Do or Ving Tsun of software development; since it contains almost no form in itself; it's very easy embrace and extend into whatever you fancy, and therefore very easy to corrupt.
  - lancerkind 9y ago
    I'm curious how much time was spent trying to write test driven code before deciding it wasn't working for you. Did you succesfully write a few small programs using TDD (coding katas)? Or did you apply TDD to your project at work? For how many hours or lines of code did you try TDD before deciding it wasn't going to work?

eesmith 9y ago

TDD doesn't really seem to be useful, at least, not as clearly useful as its proponents generally advocate. That is, it's generally advocated as a clear win, but of the various attempts to quantify it, it's a mix at best. For example, https://www.computer.org/csdl/trans/ts/2013/06/tts2013060835.html https://www.computer.org/csdl/trans/ts/2013/06/tts2013060835... says "This paper provides a systematic meta-analysis of 27 studies that investigate the impact of Test-Driven Development (TDD) on external code quality and productivity. The results indicate that, in general, TDD has a small positive effect on quality but little to no discernible effect on productivity." It then goes on about subgroup analysis, but we must take that with a grain of salt as subgroup analysis is more likely to have "statistically significant" correlations due to random fluctuations. When I try TDD on my own projects, I find that I lock in the internal API too quickly, that is, I test against internal functions, because they are easy to test. Then when I refactor, I find I avoid refactoring which might, for example, merge a couple of functions into one or otherwise change that internal API, because of the effort of rewriting the tests to meet the new API. I work best with "Spike and Stabilize", that is, a spike solution, flesh it out, then once the API is mostly finished, develop the tests. I also use coverage testing to help identify which tests are missing. Side note: I hate it when TDD people say that TDD code naturally has 100% coverage. It doesn't. At a process level, the refactor can include "replace algorithm", but the new algorithm may have different edge cases than the original algorithm, so require additional tests to ensure correctness. For example, replace a simple quicksort with timsort, because it's possible to build a test case which causes quicksort to hit quadratic worst-case performance. But if the input has 63 elements or fewer then only the insertion sort part of timsort will be tested. You need a very different test suite to really exercise timsort, compared to what's needed to test a simple quicksort. Or for a simpler example, performance analysis may say that the case n=1 takes 90% of the time, so rather than use the slower general-purpose algorithm, the new strategy is to special-case n=1. However, deeper in the code there was already special support for n=1, which is now no longer used. There's no way TDD can identify this dead code. Has there been any analysis of coverage analysis on TDD-based projects, where coverage was never evaluated before, to see what the coverage actually is?

lancerkind 9y ago
Regarding coverage comment, your right in that code can contain capabilities (or bugs) beyond the TDD coded conceived. (If they weren't conceived then it could be a bug or things that don't happen in practical operations.) The TDD people are right in that if you follow the TDD process, all your code comes up as covered by a code coverage tool. But this is only coverage. Not all possible functionality coverage. I imagine "all possible functionality coverage" is an intractable problem in the same class as the Halting Problem. Branch/line coverage should be near 100% for TDD code. (Some I/O coupling areas are intentionally saved for other types testing as they aren't unit testable.) It's an easy thing to measure. It's a common industry metric. It's not the whole picture but it is something. Measuring code coverage on TDD created software that was built without monitoring the code coverage, then later use code coverage at the end as an evaluation: I've only a few anecdotal (two) instances where I did this and I found the code coverage to be well above 95% for the code base and at 100% for modules that were pure logic. In one of those instances, after many rounds of defect injection, I discovered one or two locations where another unit test could be added. All in all, I've been pretty happy with TDD. But who cares? I want to hear from the industry why others aren't doing it. :-) Thank you eesmith, I enjoyed the discussion! How about hearing some more from others? What's stopping you from doing TDD? Is it because you don't believe it? If you don't believe it, how was it presented to you? Did you try it?