5 ms·

The conversation around TDD is tired, and the conclusion is always the same: "it depends." It depends on the person writing the code, the type of problem they'r

The conversation around TDD is tired, and the conclusion is always the same: "it depends." It depends on the person writing the code, the type of problem they're tackling, the language they're using, the needs of the business, etc. This study doesn't bring anything new to the table except: "in this manufactured environment we found a single point of data that equates to noise." I'm guessing the only reason the story was upvoted at all in the first place is because some people who agree with the title clicked the up arrow without looking at the article.

tbrownaw 10y ago
the conclusion is always the same: "it depends." Er, no. The studies I've read all end up showing that principled testing helps, but test-first and TDD (strict red/green cycle, code only enough to pass the new test, etc) provide no additional benefit over anything else that gets the tests written. The "it depends" always comes from the echo chamber trying to justify their desire to believe that TDD isn't completely useless. It actually feels quite similar to the claims I've seen from practitioners that reikei, faith healing, etc aren't complete bunk.
- bjelkeman-again 10y ago
  > provide no additional benefit over anything else that gets the tests written. Isn't this enough though? The tests gets written, which probably was the main point in the first place?
  - zorked 10y ago
    You can also add a coverage tool to your CI and get the same result (tests get written) without any of the ideology (TDD fairies sprinkle unicorn dust everywhere).
    - _ikke_ 10y ago
      Coverage tools are not sufficient for getting good test coverage. One can easily make code 'covered' without having proper tests for them. By writing tests early, you make sure the code you are testing is testable and your knowledge about the code is fresh.
      - lomnakkus 10y ago
        > By writing tests early, you make sure the code you are testing is testable and your knowledge about the code is fresh. IME you also end up writing tests which are far too tied to the implementation. (With the resulting churn that that implies when the implementation changes.) You get far more mileage from QuickCheck-type tests IME. Granted, not everything is very amenable to testing using QC-type tests, but a lot of stuff is.
    - vidarh 10y ago
      That assumes you are disciplined enough to act on the result. Which is what most of these mechanism are about: Finding what causes sufficient friction to get the tests written. (I'm making this comment because I know of a team with a coverage tool tied into CI where the coverage has been ignored for years)
    - dllthomas 10y ago
      I've heard that tests written specifically to drive up coverage metrics aren't much better than no tests.
- ericdykstra 10y ago
  I really don't believe any study or meta-study could come close to being able to suss out the nuance of when TDD may provide an advantage and when it doesn't. I'd rather just trust programmers to consider what approach works best for their problem and mindset and go from there. I personally don't TDD most things, but it's a tool I have available and I bring it out when a situation arises.
  - raverbashing 10y ago
    Well, with TDD you waste time writing red tests first, then with tests that "just pass" and finally make the thing as it is supposed to be I find it surprising that's not slower than Test-Last (maybe if you really leave it for last then you'll need some time to fit your functions to your test)
  - dllthomas 10y ago
    I think we could, but it'd be very expensive (multiple large and careful studies) and only possibly worth it. > I'd rather just trust programmers to consider what approach works best for their problem and mindset and go from there. People are often surprisingly good at fooling themselves. I'd rather have actual empirical validation. And also a pony.
djsumdog 10y ago
You know, this study and others don't take about long term maintainability or brekability. In fact, isn't this a good thing? If it has little impact on development time, we should totally be doing it. That way, we have tests. We have them from the start. We met the conditions we wanted instead of guessing conditions at the end.

tbrownaw 10y ago

the conclusion is always the same: "it depends." Er, no. The studies I've read all end up showing that principled testing helps, but test-first and TDD (strict red/green cycle, code only enough to pass the new test, etc) provide no additional benefit over anything else that gets the tests written. The "it depends" always comes from the echo chamber trying to justify their desire to believe that TDD isn't completely useless. It actually feels quite similar to the claims I've seen from practitioners that reikei, faith healing, etc aren't complete bunk.

bjelkeman-again 10y ago
> provide no additional benefit over anything else that gets the tests written. Isn't this enough though? The tests gets written, which probably was the main point in the first place?
- zorked 10y ago
  You can also add a coverage tool to your CI and get the same result (tests get written) without any of the ideology (TDD fairies sprinkle unicorn dust everywhere).
  - _ikke_ 10y ago
    Coverage tools are not sufficient for getting good test coverage. One can easily make code 'covered' without having proper tests for them. By writing tests early, you make sure the code you are testing is testable and your knowledge about the code is fresh.
    - lomnakkus 10y ago
      > By writing tests early, you make sure the code you are testing is testable and your knowledge about the code is fresh. IME you also end up writing tests which are far too tied to the implementation. (With the resulting churn that that implies when the implementation changes.) You get far more mileage from QuickCheck-type tests IME. Granted, not everything is very amenable to testing using QC-type tests, but a lot of stuff is.
  - vidarh 10y ago
    That assumes you are disciplined enough to act on the result. Which is what most of these mechanism are about: Finding what causes sufficient friction to get the tests written. (I'm making this comment because I know of a team with a coverage tool tied into CI where the coverage has been ignored for years)
  - dllthomas 10y ago
    I've heard that tests written specifically to drive up coverage metrics aren't much better than no tests.
ericdykstra 10y ago
I really don't believe any study or meta-study could come close to being able to suss out the nuance of when TDD may provide an advantage and when it doesn't. I'd rather just trust programmers to consider what approach works best for their problem and mindset and go from there. I personally don't TDD most things, but it's a tool I have available and I bring it out when a situation arises.
- raverbashing 10y ago
  Well, with TDD you waste time writing red tests first, then with tests that "just pass" and finally make the thing as it is supposed to be I find it surprising that's not slower than Test-Last (maybe if you really leave it for last then you'll need some time to fit your functions to your test)
- dllthomas 10y ago
  I think we could, but it'd be very expensive (multiple large and careful studies) and only possibly worth it. > I'd rather just trust programmers to consider what approach works best for their problem and mindset and go from there. People are often surprisingly good at fooling themselves. I'd rather have actual empirical validation. And also a pony.

djsumdog 10y ago

You know, this study and others don't take about long term maintainability or brekability. In fact, isn't this a good thing? If it has little impact on development time, we should totally be doing it. That way, we have tests. We have them from the start. We met the conditions we wanted instead of guessing conditions at the end.