4 ms·

This is the opinion of a single engineer on a single team. There are tens of thousands other engineers and thousands of other teams. My team (at Microsoft) is

This is the opinion of a single engineer on a single team. There are tens of thousands other engineers and thousands of other teams. My team (at Microsoft) is religious about unit tests for some parts of the product (math libraries, content serialization logic, etc) and have a grand total of zero "unit tests" for other components (some UI, etc). We do, however, have a full test team who write numerous automated regression tests as well as conduct significant manual testing and routinely use the product. Furthermore, have you read it? It's not saying don't write tests. It's saying that TDD is not a silver bullet.

rbanffy 17y ago
I am sorry. I was being cruel. It may not be a silver bullet, but it's the kind of "don't leave home without it" ammo any self-respecting programmer will keep close all the time. Without them the risk of regressions is immense.
- snprbob86 17y ago
  You're making far too many assumptions about the code to be tested. The folks who wrote our math libraries (primitives such as 4x4 matrices, vectors, planes, spheres, axis-aligned boxes, etc), would agree "don't leave home without it" for TDD. The folks who wrote our drawing libraries cheat on the TDD a bit and write the code first, then capture a screenshot and use that screenshot for automated testing to verify nothing breaks in the future. The folks who wrote the sample code that implements a simple game (involving math and graphics) laugh at the concept of TDD. What would you possibly test? There is no `bool IsFun(Game)` function. How should Mario jump? I dunno, just pick some random numbers and math operations until it feels like it works. Then try not to break it. The interesting bit here is that each of these groups of folks are on the same team, in fact many are the same people. Use the right tool for the job. Invest your testing efforts where you get the most bang for buck.
  - rbanffy 17y ago
    Using a screen capture is not exactly cheating. It's automating regression testing and that's a Good Thing. A human ensures the first implementation is correct and then the tests ensure future implementations are correct too. There is also stuff that could (and should) be tested - like a drawing primitive that should not exceed a given region, a pixel that should be of a given color or even an operation that should finish in a given number of cycles for a given pixel count. Erroneous arguments should also be tested as the proper exceptions should be thrown (or, at the very least, proper error codes should be emitted). For those who write the demo code, there is a lot of tests to do on the gameplay itself - collisions detection, rendering tests, physics and a whole lot of stuff that will also serve as nice documentation on how to use the components that are being exercised.
    - snprbob86 17y ago
      TDD says "no line of product code is written until there is a unit test that fails for it". I'm just pointing out that there is a spectrum of practicality in TDD. Our automated regression tests for graphics routinely require humans to re-verify every single screen capture because a single pixel may render slightly differently with some other change and there is no algorithm to determine "is this artifact objectionable to humans". Tests are good, but they are not a panacea. As for the "proper exceptions" or error codes or what not. That always drove me NUTS to see that. The first line of the function is `if (foo == null) throw new ArgumentNullException("foo")` -- do I really need to write a unit test for that? Furthermore, static analysis tools tell us when your public interface (including exception types we may throw) changes, that is code reviewed too. Writing `AssertThrows(typeof(ArgumentNullException)...` is just a waste of time. Again: this is not an argument about whether tests have merit or not. All the author (and I) are saying, at the core of our argument, is that write tests when it makes sense to write tests. He's also going a step further to say "it makes sense to write tests less frequently than TDD nuts would have you believe". I may or may not agree with him, but I'm going to publicly denounce that belief simply because I'd rather developers write too many tests than too few.
      - tesseract 17y ago
        > TDD says "no line of product code is written until there is a unit test that fails for it". If you have tests that completely specify the behavior of the code, it seems a bit redundant to then write the code... why not automatically generate it? Have I just described, in some vague sense, prolog?

rbanffy 17y ago

I am sorry. I was being cruel. It may not be a silver bullet, but it's the kind of "don't leave home without it" ammo any self-respecting programmer will keep close all the time. Without them the risk of regressions is immense.

snprbob86 17y ago
You're making far too many assumptions about the code to be tested. The folks who wrote our math libraries (primitives such as 4x4 matrices, vectors, planes, spheres, axis-aligned boxes, etc), would agree "don't leave home without it" for TDD. The folks who wrote our drawing libraries cheat on the TDD a bit and write the code first, then capture a screenshot and use that screenshot for automated testing to verify nothing breaks in the future. The folks who wrote the sample code that implements a simple game (involving math and graphics) laugh at the concept of TDD. What would you possibly test? There is no `bool IsFun(Game)` function. How should Mario jump? I dunno, just pick some random numbers and math operations until it feels like it works. Then try not to break it. The interesting bit here is that each of these groups of folks are on the same team, in fact many are the same people. Use the right tool for the job. Invest your testing efforts where you get the most bang for buck.
- rbanffy 17y ago
  Using a screen capture is not exactly cheating. It's automating regression testing and that's a Good Thing. A human ensures the first implementation is correct and then the tests ensure future implementations are correct too. There is also stuff that could (and should) be tested - like a drawing primitive that should not exceed a given region, a pixel that should be of a given color or even an operation that should finish in a given number of cycles for a given pixel count. Erroneous arguments should also be tested as the proper exceptions should be thrown (or, at the very least, proper error codes should be emitted). For those who write the demo code, there is a lot of tests to do on the gameplay itself - collisions detection, rendering tests, physics and a whole lot of stuff that will also serve as nice documentation on how to use the components that are being exercised.
  - snprbob86 17y ago
    TDD says "no line of product code is written until there is a unit test that fails for it". I'm just pointing out that there is a spectrum of practicality in TDD. Our automated regression tests for graphics routinely require humans to re-verify every single screen capture because a single pixel may render slightly differently with some other change and there is no algorithm to determine "is this artifact objectionable to humans". Tests are good, but they are not a panacea. As for the "proper exceptions" or error codes or what not. That always drove me NUTS to see that. The first line of the function is `if (foo == null) throw new ArgumentNullException("foo")` -- do I really need to write a unit test for that? Furthermore, static analysis tools tell us when your public interface (including exception types we may throw) changes, that is code reviewed too. Writing `AssertThrows(typeof(ArgumentNullException)...` is just a waste of time. Again: this is not an argument about whether tests have merit or not. All the author (and I) are saying, at the core of our argument, is that write tests when it makes sense to write tests. He's also going a step further to say "it makes sense to write tests less frequently than TDD nuts would have you believe". I may or may not agree with him, but I'm going to publicly denounce that belief simply because I'd rather developers write too many tests than too few.
    - tesseract 17y ago
      > TDD says "no line of product code is written until there is a unit test that fails for it". If you have tests that completely specify the behavior of the code, it seems a bit redundant to then write the code... why not automatically generate it? Have I just described, in some vague sense, prolog?