5 ms·

As any rant it needs to be taken with a grain of salt. This one, however, is very true: > I and everyone I have ever talked to about testing has had the same

As any rant it needs to be taken with a grain of salt. This one, however, is very true: > I and everyone I have ever talked to about testing has had the same experience; we write a new feature, test the hell out of it, can’t find anything wrong; ask a coworker to take a crack at it, and he finds a major bug in under a minute. The password is blind spots, and we all have them. And if we didn’t think of some case in development, we are not going to think of it in testing. That’s why we had separate QA departments. I wouldn't go as far as the author saying TTD is useless as it's genuinely useful against most obvious bugs, but it's only a a part of the equation. But it's not like all companies removed their QA departments either - many pieces of software are regularly tested by independent teams with good results. If someone thinks TTD will eliminate all bugs, well, they're not thinking clearly.

js8 5y ago
> I wouldn't go as far as the author saying TTD is useless as it's genuinely useful against most obvious bugs I am skeptical of TDD, so I have a genuine question. Let's say TDD is able to find the "most obvious" bugs, like a typo. My question is, why are these bugs worth creating a test for them, compared to either manual testing (running the program and looking at the output) or running a portion of the integrated test suite on the code to find them? If the bug is simple and crippling, then the program is likely to fail anyway. To me there is a tradeoff. With TDD, you write some unit test code, and it finds a trivial bug. But is every trivial bug found worth maintaining the unit test code going forward? I mean it's not like this bug is gonna appear again all of sudden in your code, unless you change it, but then in all likelihood you need to change your unit test as well. I guess I don't really see TDD as a good tool, because all the situations that you can run into are already covered by other, more specialized tools. If you want to catch edge cases, asserts are IMHO superior tool to tests. If you want to verify your program quickly, running it manually and looking at the output is superior. If you want to verify your program properly, running a comprehensive (and integrated) test suite is superior (the best if you can make it property-based). There are situations where unit tests are genuinely useful, like writing a library. But in those cases, unit tests seem to always either compare with some other implementation (for example, if I am testing a sin(x) function, my other implementation is the calculator), or at least compare with well-defined specification that comes outside the code. But I feel like majority of tests written for TDD are not really testing the code against anything (simply because another implementation or specification is not available), they just test the intent of the code itself as is written, which might be useful if you wanted to write "bar" but wrote "baz" by accident, but other than this type of bugs, what is their long-term contribution, that could justify the maintenance cost?
- loopz 5y ago
  The output of TDD is not the tests themselves, but the process of designing beforehand what is expected behaviour using up-front tests. The tests then acts as a guide to later development of code. Both should be adjusted during the discovery journey. The tests should then reflect the value that the code provides, and must be usable as-is in later refactors to have much worth themselves. "Acceptance criterias" should be self-evident from the ongoing development processes, and is not a final signoff. TDD tests would ideally be used for hunting business value, and focus on overall desired product behaviour and qualities.
  - js8 5y ago
    Are you saying you want to replace the specification with tests? That's really dubious to me. I don't think it really helps software development, if instead of "here's the description of what we need to do" the SW developer gets "here's a few examples of what we need to do" (albeit if the examples can be automatically verified). If that's NOT your intent, what's the role of specification in the TDD, and how it's interacting with the implementation and the tests?
    - loopz 5y ago
      It's just the idea behind Test Driven Development, that tests drive development. It won't be very effective as a handover, though could be used as acceptance criterias if that is needed. The adherents really want tests to be the specification, as a way to be measurable / testable. Specs may be converted to tests. But most of it is theory. It all depends, as usual, on the dev(s) and circumstances.
      - fatnoah 5y ago
        My one and only truly TDD project went brilliantly, but there were specific circumstances. In this project, I created the back-end and API while an outsourced team built the UI. What made it work well was that I wrote the tests AND the spec at the same time, and kept both up to date. Completed tests equalled progress on the back-end, made it easy to identify any regressions in my own code, and clear differentiated between logic and general HTTP plumbing errors.
JKCalhoun 5y ago
> I wouldn't go as far as the author saying TTD is useless as it's genuinely useful against most obvious bugs The obvious bugs should already be accounted for if you have decent param checking in most functions. Is the string empty? Is the array nil? Did the init succeed? Inline unit tests anyone? :-)

js8 5y ago

> I wouldn't go as far as the author saying TTD is useless as it's genuinely useful against most obvious bugs I am skeptical of TDD, so I have a genuine question. Let's say TDD is able to find the "most obvious" bugs, like a typo. My question is, why are these bugs worth creating a test for them, compared to either manual testing (running the program and looking at the output) or running a portion of the integrated test suite on the code to find them? If the bug is simple and crippling, then the program is likely to fail anyway. To me there is a tradeoff. With TDD, you write some unit test code, and it finds a trivial bug. But is every trivial bug found worth maintaining the unit test code going forward? I mean it's not like this bug is gonna appear again all of sudden in your code, unless you change it, but then in all likelihood you need to change your unit test as well. I guess I don't really see TDD as a good tool, because all the situations that you can run into are already covered by other, more specialized tools. If you want to catch edge cases, asserts are IMHO superior tool to tests. If you want to verify your program quickly, running it manually and looking at the output is superior. If you want to verify your program properly, running a comprehensive (and integrated) test suite is superior (the best if you can make it property-based). There are situations where unit tests are genuinely useful, like writing a library. But in those cases, unit tests seem to always either compare with some other implementation (for example, if I am testing a sin(x) function, my other implementation is the calculator), or at least compare with well-defined specification that comes outside the code. But I feel like majority of tests written for TDD are not really testing the code against anything (simply because another implementation or specification is not available), they just test the intent of the code itself as is written, which might be useful if you wanted to write "bar" but wrote "baz" by accident, but other than this type of bugs, what is their long-term contribution, that could justify the maintenance cost?

loopz 5y ago
The output of TDD is not the tests themselves, but the process of designing beforehand what is expected behaviour using up-front tests. The tests then acts as a guide to later development of code. Both should be adjusted during the discovery journey. The tests should then reflect the value that the code provides, and must be usable as-is in later refactors to have much worth themselves. "Acceptance criterias" should be self-evident from the ongoing development processes, and is not a final signoff. TDD tests would ideally be used for hunting business value, and focus on overall desired product behaviour and qualities.
- js8 5y ago
  Are you saying you want to replace the specification with tests? That's really dubious to me. I don't think it really helps software development, if instead of "here's the description of what we need to do" the SW developer gets "here's a few examples of what we need to do" (albeit if the examples can be automatically verified). If that's NOT your intent, what's the role of specification in the TDD, and how it's interacting with the implementation and the tests?
  - loopz 5y ago
    It's just the idea behind Test Driven Development, that tests drive development. It won't be very effective as a handover, though could be used as acceptance criterias if that is needed. The adherents really want tests to be the specification, as a way to be measurable / testable. Specs may be converted to tests. But most of it is theory. It all depends, as usual, on the dev(s) and circumstances.
    - fatnoah 5y ago
      My one and only truly TDD project went brilliantly, but there were specific circumstances. In this project, I created the back-end and API while an outsourced team built the UI. What made it work well was that I wrote the tests AND the spec at the same time, and kept both up to date. Completed tests equalled progress on the back-end, made it easy to identify any regressions in my own code, and clear differentiated between logic and general HTTP plumbing errors.

JKCalhoun 5y ago

> I wouldn't go as far as the author saying TTD is useless as it's genuinely useful against most obvious bugs The obvious bugs should already be accounted for if you have decent param checking in most functions. Is the string empty? Is the array nil? Did the init succeed? Inline unit tests anyone? :-)