19 ms·

I'm not sure what you mean by "TDD is the Cause of the bugs" - the TDD projects had 40-90% fewer bugs. Even if their remaining bugs would have been caught by te

by newt 16y ago

I'm not sure what you mean by "TDD is the Cause of the bugs" - the TDD projects had 40-90% fewer bugs. Even if their remaining bugs would have been caught by test coverage of things that occur in live use, that's still a good result, right?

JoeAltmaier 16y ago
Well, sort of. TDD defines bugs as "things the unit tests catch". And the unit tests are very narrow. So yes, you record fewer bugs, fix those narrow cases quickly and pretend you're done. Then it gets to the field, and whatever you didn't catch becomes a 3-alarm fire. I vastly prefer shakedown tests of the sort where you launch a 'bot army' on cloud servers to thrash the app. Especially if there is Any network/client/server component.
- newt 16y ago
  TDD defines bugs as "things the unit tests catch". And the unit tests are very narrow. I think that if you read the article linked, you'd find that the study didn't work that way. Any study that measures different things between experiment and control groups is not going to be sound. Then it gets to the field, and whatever you didn't catch becomes a 3-alarm fire. I've worked on TDD projects, and those are definitely are counted as bugs. Severity 1.
  - steveklabnik 16y ago
    ... and then you get a regression test, so that bug never happens again. And the regression test is easy to write, since all of your code is already testable, you have all the testing infrastructure in place, etc.
    - JoeAltmaier 16y ago
      Never happens again? You already have the 3-alarm fire, its too late. That's my point exactly. You catch only what you can think of and write tests to cover.
      - newt 16y ago
        I'm not sure how TDD makes this worse. Unless you think that it absolves you of any end-to-end QA testing. In which case, welcome to the fire.
        JoeAltmaier 16y ago
        I object to the TDD fundamental precept that you code to the tests. That narrows the 'testing' until its a self-fulfilling prophecy that you will have 'fewer bugs', since you defined the bugs as 'what we're looking for'. Reminds me of the old joke "A: I'm looking for my wallet. B: where did you lose it? A: over there. B: then why look here? A: the light's better"
        newt 16y ago
        You're repeating your false idea that bugs in TDD come only from the unit tests. Eh, see comments above.

by newt 16y ago

JoeAltmaier 16y ago

Well, sort of. TDD defines bugs as "things the unit tests catch". And the unit tests are very narrow. So yes, you record fewer bugs, fix those narrow cases quickly and pretend you're done. Then it gets to the field, and whatever you didn't catch becomes a 3-alarm fire. I vastly prefer shakedown tests of the sort where you launch a 'bot army' on cloud servers to thrash the app. Especially if there is Any network/client/server component.

newt 16y ago
TDD defines bugs as "things the unit tests catch". And the unit tests are very narrow. I think that if you read the article linked, you'd find that the study didn't work that way. Any study that measures different things between experiment and control groups is not going to be sound. Then it gets to the field, and whatever you didn't catch becomes a 3-alarm fire. I've worked on TDD projects, and those are definitely are counted as bugs. Severity 1.
- steveklabnik 16y ago
  ... and then you get a regression test, so that bug never happens again. And the regression test is easy to write, since all of your code is already testable, you have all the testing infrastructure in place, etc.
  - JoeAltmaier 16y ago
    Never happens again? You already have the 3-alarm fire, its too late. That's my point exactly. You catch only what you can think of and write tests to cover.
    - newt 16y ago
      I'm not sure how TDD makes this worse. Unless you think that it absolves you of any end-to-end QA testing. In which case, welcome to the fire.
      - JoeAltmaier 16y ago
        I object to the TDD fundamental precept that you code to the tests. That narrows the 'testing' until its a self-fulfilling prophecy that you will have 'fewer bugs', since you defined the bugs as 'what we're looking for'. Reminds me of the old joke "A: I'm looking for my wallet. B: where did you lose it? A: over there. B: then why look here? A: the light's better"
        newt 16y ago
        You're repeating your false idea that bugs in TDD come only from the unit tests. Eh, see comments above.