5 ms·

It's not a good argument against testing, I agree. Automated testing is a huge boon for software quality (though by no means a panacea). But TDD, in my experie

• ncphillips 11y ago
  > Because you generated the code being motivated by the tests, there are a lot of unit tests that dot test functional units - tests that are essentially testing implementation decisions. I've seen this happen to myself numerous times as well, but I don't think this is good argument against TDD. If a tests fails because the implementation details change, that tells us that we wrote a bad test, not that TDD is bad. As I get better at writing tests, I am better able to catch these kinds of errors ahead of time. There are a ton of little techniques you can use and questions you can ask yourself about your code that can make your tests better. Furthermore, test code is just as important as production code, and we should be just as ruthless in refactoring and peer-reviewing each others tests (within reason). TDD is NOT suitable for every problem, this is true. But I think too many people give up on it because they see "the problems of TDD" which are actually just "the problems of writing bad tests". Note: I'm not trying to target you sago, but merely making a general observation. I'm talking more about people who don't write tests even close to the time of writing the code, or don't write any tests at all because the their tests kept breaking for no good reason and were slowing them down.
  • sago 11y ago
    > that tells us that we wrote a bad test, not that TDD is bad I strongly disagree. The only thing that matters to how software works for an end user is its boundary. You simply can't do TDD by only testing the boundary. You have to write a test before each unit of code, so you have to test those internal interfaces, the way units of code work. But that is exactly the stuff that should be allowed to change. It's a mistake to assume 'implementation details' only exist within a function. That stuff you can avoid testing, but the behavior of the function is itself an implementation detail for whatever is using that function. I simply cannot imagine TDD for anything other than a toy example where you don't end up writing tests that depend on the software's implementation, and are therefore inertia to changing that. > we should be just as ruthless in refactoring and peer-reviewing each others tests I strongly agree. But this is rarely part of the TDD ethos. Saying "I refactored the code and threw away 100 tests." is likely to send TDD folks twitching, in my experience.
    • mbrock 11y ago
      Changing tests is work, yes. Unit tests are precisely about verifying implementations, and maintaining them is the price you pay for having these layers of verification. It's inertia in the way a safety net is a barrier.
      • hinkley 11y ago
        I have always found the 'safety net' analogy uninspiring, and people who think they're tough and manly don't need safety nets, thankyouverymuch. Now, safety equipment in a race car is something else entirely. Half of it keeps you from dying immediately on impact, sure, but the other half actually lets you go faster. The 5 point harness, for example, helps keep you from losing control of the vehicle during high G maneuvers, by keeping your torso in the seat and therefore your legs and arms in (roughly) the correct orientation to the steering wheel and the pedals.
    • firepoet 11y ago
      Not this TDD folk. :-)
    • RHSeeger 11y ago
      > The only thing that matters to how software works for an end user is its boundary. You're ignoring the fact that tests aren't only to validate "end user functionality". They're also there to make life easier for the developer; to make both initial development and maintenance easier. If I need a custom sort algorithm for my product, I'm going to write unit tests for it's implementation. I'm also going write functional tests to make sure the end user is seeing things sorted. However, those tests serve different purposes. The unit tests allow me to write clean, simple tests that show exactly what the implementation is supposed to be doing for different inputs. That allows me to change the implementation of the sorting algorithm later and worry less about breaking the functionality. The functional tests are (generally) not going to be as clean and obvious, nor cover every possible partition of inputs that the implementation could get.
      • sago 11y ago
• crdoconnor 11y ago
  >I've seen situations where that big refactoring would involve scrapping or rewriting tests, and that causes its own kind of architectural conservatism. I call this test concreting: https://hitchtest.readthedocs.org/en/latest/glossary/test_concreting.html https://hitchtest.readthedocs.org/en/latest/glossary/test_co... The way to avoid it is to either be damn sure that the API you've surrounded is tight, clean and unlikely to need to change much or to test at a higher level.
• atemerev 11y ago
  > tests that are essentially testing implementation decisions. There shouldn't be any. Those are not useful at all.
  • sago 11y ago
    See my reply above. I think your view is either naive, or you understand 'implementation decision' in too narrow a way.
    • atemerev 11y ago
      I understand "implementation decisions" in quite well-defined way; only functional invariants should be tested. Your example is a perfect illustration for one. Additional point is that invariant-based testing can easily be transformed into automated fuzzy tests using Quickcheck and its analogues, and this can find immense amount of otherwise hidden bugs.
      • sago 11y ago
        > only functional invariants should be tested. That makes no sense at all. > Your example is a perfect illustration I didn't give an example. > invariant-based testing can easily be transformed into automated fuzzy tests 'fuzz' not 'fuzzy'. True, but irrelevant to the point being argued.
• maxxxxx 11y ago
  "I've seen situations where that big refactoring would involve scrapping or rewriting tests, and that causes its own kind of architectural conservatism." Yes, rewriting tests is part of a refactor. You always have to be willing to do that or just scrapping outdated tests.
• geoelectric 11y ago
  I come to this from a test/automation engineering background and I'd like to amplify your last point. There are a couple things to consider that sometimes become taboo in orgs--but shouldn't. The first is that you don't need to test everything. I recommend having a great set of those API-level black box tests as an acceptance suite, but at a component or unit level you should pick your battles. If you'd architect for DI -only- to test, that testing had better be pretty necessary, otherwise you introduced complication (and risk) without a lot of benefit. That risk can actually overwhelm any benefit of testing. The second is that you should absolutely scrap tests of implementation details at the point that the implementation becomes a black box. Tests are like scaffolding during construction, and we don't leave the scaffolding up when the building is done--we do inspections of the building instead. The same mindset should be applied to testing. Different phases need different tests, and too many tests at too low a level of abstraction absolutely cause maintenance and flexibility issues if you're not willing to let them go. But that's no reason to back off entirely. SOME things need to be unit- or component-tested SOMETIMES. Those things generally benefit from TDD. It's all about making intelligent decisions. There's a sometimes-controversial movement in QA/test called "context-driven testing" which comes down to there are very few always-best practices, and strategy should be tailored to your problem and situation--not done by rote, book, or prior (read, your boss's or CTO's) assumptions of what's always right. Otherwise you tend to get something more onerous than useful. The considerations around that approach apply at the low-level test tier as well.