4 ms·

I've written so many automated tests, but how many have actually caught something important? A couple dozen? Yeah I'm glad TDD isn't the dominant religion any

I've written so many automated tests, but how many have actually caught something important? A couple dozen? Yeah I'm glad TDD isn't the dominant religion any more. If you write a test and it never fails throughout the lifetime of the project, thats wasted time.

moron4hire 4y ago
The last time I found TDD to be useful was when I was implementing some code for processing of data feeds according to a pretty well-defined specification. The spec translated to tests quite easily and then the tests very readily pointed out where the data did not match the spec 100%. Luckily, we were in a position to modify the data-generating side of things, too, to make sure everything was in-spec. But most of the time, I'm dealing with user interfaces, service integrations, and data munging. I have "test benches" instead of unit tests for these. They're like "minimal reproductions", fully stand-alone programs that exercise the code in the "expected" way. The big issue is that the "expected way" could change at practically any minute, depending on business need. There's no "spec" other than, "are people's expectations for a usable, comfortable system being met?" We want them to be changeable at the drop of a hat. And those test benches help make those changes possible, by not having to work in the full UI to change the behavior of one control, one service, etc.
dllthomas 4y ago
I want to push back on this in two ways. First, it's my understanding (as someone who usually doesn't TDD and hasn't read/thought that deeply about it in a while) that the core of TDD (vs other testing doctrines) was to iterate by making a failing test, then making it pass. If that's correct, then if you're writing tests that never fail throughout the lifetime of the project then you're not doing TDD, though culture does weird things to practices and definitions and I'm sure there's places that called any damned thing TDD when it had more hype. Second, I think a test (or static check) that never fails can be useful if it lets the programmer pay less attention to making sure the program is correct in that particular way (freeing up bandwidth for other important considerations), and as a communication tool demonstrating that the system does in fact work the way it does. Whether those benefits are worth it depends on a bunch of features of the particular context, including how long it takes to write/maintain the test (although if it never fails then at least you probably don't need to update it) and how long it takes to run the test.
- qudat 4y ago
  > If that's correct, then if you're writing tests that never fail throughout the lifetime of the project then you're not doing TDD Agreed. > [tests that never fail can be useful] as a communication tool demonstrating that the system does in fact work the way it does. Agreed as well. That's what makes the section I wrote on testing sting for me. I read about all the benefits and see how it can be helpful as a form of documentation. I see all the upsides but at the end of the day, we still need to calculate if automated tests are really worth it. The maintenance burden on them can be huge. A couple line change and confirming it works with a manual test can take a couple of minutes, but fixing an integration test that is now failing can take the entire day or worse. It's tough, but my main goal was to challenge the dogma of "always write automated tests."
  - justin_oaks 4y ago
    > It's tough, but my main goal was to challenge the dogma of "always write automated tests." It's perfectly fine to challenge dogma and ask people figure out whether the automated tests make sense for their use case. I think the push for automated tests came when there were so many others who grossly underestimate the benefits of automated tests and dismiss them off-hand. It's hard to see the benefits of automated testing. It's easy to see the time spent writing the tests while it's hard to see the time saved by catching bugs before they happen. It's also easier to refactor if you have tests to confirm that your refactoring didn't break something.
  - dllthomas 4y ago
    Right, just because it has value doesn't mean it has enough value. My objection to the parent (not the article) was that it wound up dismissing certain kinds of value.
- codeulike 4y ago
  the core of TDD was to iterate by making a failing test, then making it pass Yes and when I first heard that 15 years ago (or whenever it was) it sounded like genius and I did it on a few projects. Its a useful crutch maybe if you're doing something very unfamiliar But as a dogma that must always be followed? And having a test for every single 'specification' and edge case? No - screw that, you end up faffing around for ages setting up mocks and testing stuff that doesnt deserve it. And you end up with a massive library of tests that get on your nerves. Also everything got completely bent out of shape for a few years because everything had to be injectable and testable. https://dhh.dk/2014/tdd-is-dead-long-live-testing.html https://dhh.dk/2014/tdd-is-dead-long-live-testing.html http://web.archive.org/web/20180215225218/https://iansommerville.com/systems-software-and-technology/giving-up-on-test-first-development/ http://web.archive.org/web/20180215225218/https://iansommerv... http://blog.cleancoder.com/uncle-bob/2016/03/19/GivingUpOnTDD.html http://blog.cleancoder.com/uncle-bob/2016/03/19/GivingUpOnTD... https://rbcs-us.com/documents/Why-Most-Unit-Testing-is-Waste.pdf https://rbcs-us.com/documents/Why-Most-Unit-Testing-is-Waste...
kazinator 4y ago
Just like if you wear a seatbelt all your life, but never have an accident, what a waste of effort. Not to mention $$$ spent on car insurance. Programmers should peer into their crystal balls and predict which code is going to need to be maintained in ways that will break it and only write regression tests for that. For all the rest, just poke at it manually in your REPL and debugger and call it tested. You heard it here.
- codeulike 4y ago
  Yes, I'll write a test for something that I know from experience is likely to give me trouble later on with regression problems Writing a test when you find a bug and then fixing the bug is also quite handy sometimes And I'll write tests upfront when there's well defined outcomes and I know the code is going to be tricky But I'm not going to write tests for dumb stuff that I know wont break
zcmack 4y ago
i think TDD is a great method, but not dogma. likewise with code coverage. being prescriptive about one or the other just leads to angry people who get fed up with the principle and rebel.
taeric 4y ago
A problem with calling it wasted time, is that implies you could have recaptured that time in some other, more productive, way. And, I'd wager odds are high that you could not have.
- codeulike 4y ago
  Why not?
  - taeric 4y ago
    There ultimately is no "reason." Time just isn't completely fungible between useful and wasteful. As much as we wish it could be.
    - codeulike 4y ago
      So there's no difference between doing things that have a point and doing things that dont have a point?
      - taeric 4y ago
        Post fact, there can be. But we are terrible at knowing one from the other ahead of time. More, we don't always have "meaningful" things to do at any given moment. This is why we don't all work to type as fast as court reporters. It would not gain us much if any meaningful time. More, you can give meaning to things. Puzzles and such.
        codeulike 4y ago
        I know what you mean But if I was managing a bunch of programmers (thankfully, I dont have to do that) I would be thinking about their productivity and which activities contributed to productivity. TDD seems like a bell curve, having some can really help with productivity and regression and bugs. But too much and it just starts to be counterproductive.

moron4hire 4y ago

The last time I found TDD to be useful was when I was implementing some code for processing of data feeds according to a pretty well-defined specification. The spec translated to tests quite easily and then the tests very readily pointed out where the data did not match the spec 100%. Luckily, we were in a position to modify the data-generating side of things, too, to make sure everything was in-spec. But most of the time, I'm dealing with user interfaces, service integrations, and data munging. I have "test benches" instead of unit tests for these. They're like "minimal reproductions", fully stand-alone programs that exercise the code in the "expected" way. The big issue is that the "expected way" could change at practically any minute, depending on business need. There's no "spec" other than, "are people's expectations for a usable, comfortable system being met?" We want them to be changeable at the drop of a hat. And those test benches help make those changes possible, by not having to work in the full UI to change the behavior of one control, one service, etc.

dllthomas 4y ago

I want to push back on this in two ways. First, it's my understanding (as someone who usually doesn't TDD and hasn't read/thought that deeply about it in a while) that the core of TDD (vs other testing doctrines) was to iterate by making a failing test, then making it pass. If that's correct, then if you're writing tests that never fail throughout the lifetime of the project then you're not doing TDD, though culture does weird things to practices and definitions and I'm sure there's places that called any damned thing TDD when it had more hype. Second, I think a test (or static check) that never fails can be useful if it lets the programmer pay less attention to making sure the program is correct in that particular way (freeing up bandwidth for other important considerations), and as a communication tool demonstrating that the system does in fact work the way it does. Whether those benefits are worth it depends on a bunch of features of the particular context, including how long it takes to write/maintain the test (although if it never fails then at least you probably don't need to update it) and how long it takes to run the test.

qudat 4y ago
> If that's correct, then if you're writing tests that never fail throughout the lifetime of the project then you're not doing TDD Agreed. > [tests that never fail can be useful] as a communication tool demonstrating that the system does in fact work the way it does. Agreed as well. That's what makes the section I wrote on testing sting for me. I read about all the benefits and see how it can be helpful as a form of documentation. I see all the upsides but at the end of the day, we still need to calculate if automated tests are really worth it. The maintenance burden on them can be huge. A couple line change and confirming it works with a manual test can take a couple of minutes, but fixing an integration test that is now failing can take the entire day or worse. It's tough, but my main goal was to challenge the dogma of "always write automated tests."
- justin_oaks 4y ago
  > It's tough, but my main goal was to challenge the dogma of "always write automated tests." It's perfectly fine to challenge dogma and ask people figure out whether the automated tests make sense for their use case. I think the push for automated tests came when there were so many others who grossly underestimate the benefits of automated tests and dismiss them off-hand. It's hard to see the benefits of automated testing. It's easy to see the time spent writing the tests while it's hard to see the time saved by catching bugs before they happen. It's also easier to refactor if you have tests to confirm that your refactoring didn't break something.
- dllthomas 4y ago
  Right, just because it has value doesn't mean it has enough value. My objection to the parent (not the article) was that it wound up dismissing certain kinds of value.
codeulike 4y ago
the core of TDD was to iterate by making a failing test, then making it pass Yes and when I first heard that 15 years ago (or whenever it was) it sounded like genius and I did it on a few projects. Its a useful crutch maybe if you're doing something very unfamiliar But as a dogma that must always be followed? And having a test for every single 'specification' and edge case? No - screw that, you end up faffing around for ages setting up mocks and testing stuff that doesnt deserve it. And you end up with a massive library of tests that get on your nerves. Also everything got completely bent out of shape for a few years because everything had to be injectable and testable. https://dhh.dk/2014/tdd-is-dead-long-live-testing.html https://dhh.dk/2014/tdd-is-dead-long-live-testing.html http://web.archive.org/web/20180215225218/https://iansommerville.com/systems-software-and-technology/giving-up-on-test-first-development/ http://web.archive.org/web/20180215225218/https://iansommerv... http://blog.cleancoder.com/uncle-bob/2016/03/19/GivingUpOnTDD.html http://blog.cleancoder.com/uncle-bob/2016/03/19/GivingUpOnTD... https://rbcs-us.com/documents/Why-Most-Unit-Testing-is-Waste.pdf https://rbcs-us.com/documents/Why-Most-Unit-Testing-is-Waste...

kazinator 4y ago

Just like if you wear a seatbelt all your life, but never have an accident, what a waste of effort. Not to mention $$$ spent on car insurance. Programmers should peer into their crystal balls and predict which code is going to need to be maintained in ways that will break it and only write regression tests for that. For all the rest, just poke at it manually in your REPL and debugger and call it tested. You heard it here.

codeulike 4y ago
Yes, I'll write a test for something that I know from experience is likely to give me trouble later on with regression problems Writing a test when you find a bug and then fixing the bug is also quite handy sometimes And I'll write tests upfront when there's well defined outcomes and I know the code is going to be tricky But I'm not going to write tests for dumb stuff that I know wont break

zcmack 4y ago

i think TDD is a great method, but not dogma. likewise with code coverage. being prescriptive about one or the other just leads to angry people who get fed up with the principle and rebel.

taeric 4y ago

A problem with calling it wasted time, is that implies you could have recaptured that time in some other, more productive, way. And, I'd wager odds are high that you could not have.

codeulike 4y ago
Why not?
- taeric 4y ago
  There ultimately is no "reason." Time just isn't completely fungible between useful and wasteful. As much as we wish it could be.
  - codeulike 4y ago
    So there's no difference between doing things that have a point and doing things that dont have a point?
    - taeric 4y ago
      Post fact, there can be. But we are terrible at knowing one from the other ahead of time. More, we don't always have "meaningful" things to do at any given moment. This is why we don't all work to type as fast as court reporters. It would not gain us much if any meaningful time. More, you can give meaning to things. Puzzles and such.
      - codeulike 4y ago
        I know what you mean But if I was managing a bunch of programmers (thankfully, I dont have to do that) I would be thinking about their productivity and which activities contributed to productivity. TDD seems like a bell curve, having some can really help with productivity and regression and bugs. But too much and it just starts to be counterproductive.