4 ms·

How do you deal with tests that end up self-referencing, testing only specific implementation details and adding no additional safety? This is the trap I always

How do you deal with tests that end up self-referencing, testing only specific implementation details and adding no additional safety? This is the trap I always fall into when I try TDD. In order to add or change a feature, you now have to write the code twice: Once to implement the change, and once to re-implement the change to assert that it does what it's supposed to. And no safety has been added, because the logic itself can still be flawed. You've just written it twice instead of once. Generally I find tests that test the controllers, where multiple units of work are attached together, to be much more useful. But TDD does not naturally lead to these kinds of tests.

thomascgalvin 7y ago
> This is the trap I always fall into when I try TDD. In order to add or change a feature, you now have to write the code twice: Once to implement the change, and once to re-implement the change to assert that it does what it's supposed to. I think you have that backwards. In TDD, the test is basically an assertive statement: when this test passes, the functionality is working as intended. You then write the implementation code, and when the tests pass, you're done. This gives you two things: one, a sense of certainty that your code is meeting the requirements, and two, it bounds your work, preventing you from writing code that you don't need to make the test pass. Now let's say we're changing a feature. Maybe if a user tried to add an out-of-stock item to their cart, the UI currently displays an error message, but now we want to add a note indicating when the item is expected back in stock. We'd just go into the test for that error message, and add statements that look for this new `div`. We would probably have to add a few cases (for example, when we have an estimated data, when we don't have an estimate, and when we've been told the product won't be restocked), but we aren't fundamentally altering the test code. These tests will initially fail, because there's no `div` with an `id` of `restock-message` or whatever. But one by one we can knock them out, adding messages for known or estimated restock dates, unknown restock dates, and permanently out-of-stock items. > And no safety has been added, because the logic itself can still be flawed. That's true, but that would also be true if you were hand-testing. You can't verify functionality you don't understand, but you can't write functionality you don't understand, either. TDD, and testing in general, isn't a magic bullet that solves all of your problems; at some point, you still have to be smart enough to write the code. But testing is still a necessary tool in the developer's toolbox, because without it, you simply cannot know that your code is doing what you think it's doing. And since you're going to be testing anyway, why not have the machine do as much of the heavy lifting for you as possible?
- jniedrauer 7y ago
  > Now let's say we're changing a feature. This is where TDD really falls apart for me. It's pretty rare, at least in the code I work on, that you add a single little bit of functionality that fits cleanly into an existing TDD function. It's much more common that entire structures changes, and the TDD tests are no longer relevant and have to be completely rewritten from scratch. So again, you're effectively just implementing the logic twice. Where you actually need it (during a refactor), it provides no safety because it just gets completely rewritten every time.
  - eyelidlessness 7y ago
    I think this is a sign that you are not designing maintainable systems in the first place. I work on reasonably complex systems where significant changes to a feature do not tend to cause huge structural changes like you describe, and where the vast majority of tests tend to remain unchanged and help detect regressions. It is entirely possible, with the right design. And in my experience, developing a comfort with TDD will help promote that kind of design. (Edit: I want to clarify that this is not meant in any kind of a judgmental way about your work or your talents, I'm sincerely trying to help identify issues so that you can hopefully become more productive and more confident in the work you produce.)
eyelidlessness 7y ago
> How do you deal with tests that end up self-referencing, testing only specific implementation details and adding no additional safety? Generally, by describing tests in plain english that express "user" intent (where "user" may be the dev at the call site, or may be an end user) rather than expressing things like return values or state changes. > In order to add or change a feature, you now have to write the code twice: Once to implement the change, and once to re-implement the change to assert that it does what it's supposed to. From this description, it doesn't sound like you're doing TDD. 1. Write a failing test (where "test" describes user intent). 2. Make that test pass with an implementation. 3. Make that implementation meet your expectations for code quality, appropriate separation of concerns, appropriate abstraction, DRY etc. (This is commonly referred to as red-green-refactor.) > And no safety has been added, because the logic itself can still be flawed. You've just written it twice instead of once. While you can always get the logic wrong, it's significantly less likely if what you're testing is intent. I really don't understand what you mean by "written it twice". > Generally I find tests that test the controllers, where multiple units of work are attached together, to be much more useful. I'm... honestly surprised by that. Those tests tend to be the most brittle, the least likely to cover all scenarios you'd like to test, the most error prone, and the hardest to understand and maintain.
- jniedrauer 7y ago
  > Those tests tend to be the most brittle While true, it's far more likely that they will help you during a refactor. If you use TDD and have a test for every function, you will more often than not throw entire functions and their tests away. A million tests that test immutable functions are all useless. But tests that fall in between TDD style test-per-function and integration tests, tend to be re-used post refactor, and give certainty that the refactor worked as expected.
  - eyelidlessness 7y ago
    I think the size of a test shouldn't be defined by the size of the functionality you're testing, but rather a unit of "user intent", as expressed at whatever layer you're testing.

thomascgalvin 7y ago

> This is the trap I always fall into when I try TDD. In order to add or change a feature, you now have to write the code twice: Once to implement the change, and once to re-implement the change to assert that it does what it's supposed to. I think you have that backwards. In TDD, the test is basically an assertive statement: when this test passes, the functionality is working as intended. You then write the implementation code, and when the tests pass, you're done. This gives you two things: one, a sense of certainty that your code is meeting the requirements, and two, it bounds your work, preventing you from writing code that you don't need to make the test pass. Now let's say we're changing a feature. Maybe if a user tried to add an out-of-stock item to their cart, the UI currently displays an error message, but now we want to add a note indicating when the item is expected back in stock. We'd just go into the test for that error message, and add statements that look for this new `div`. We would probably have to add a few cases (for example, when we have an estimated data, when we don't have an estimate, and when we've been told the product won't be restocked), but we aren't fundamentally altering the test code. These tests will initially fail, because there's no `div` with an `id` of `restock-message` or whatever. But one by one we can knock them out, adding messages for known or estimated restock dates, unknown restock dates, and permanently out-of-stock items. > And no safety has been added, because the logic itself can still be flawed. That's true, but that would also be true if you were hand-testing. You can't verify functionality you don't understand, but you can't write functionality you don't understand, either. TDD, and testing in general, isn't a magic bullet that solves all of your problems; at some point, you still have to be smart enough to write the code. But testing is still a necessary tool in the developer's toolbox, because without it, you simply cannot know that your code is doing what you think it's doing. And since you're going to be testing anyway, why not have the machine do as much of the heavy lifting for you as possible?

jniedrauer 7y ago
> Now let's say we're changing a feature. This is where TDD really falls apart for me. It's pretty rare, at least in the code I work on, that you add a single little bit of functionality that fits cleanly into an existing TDD function. It's much more common that entire structures changes, and the TDD tests are no longer relevant and have to be completely rewritten from scratch. So again, you're effectively just implementing the logic twice. Where you actually need it (during a refactor), it provides no safety because it just gets completely rewritten every time.
- eyelidlessness 7y ago
  I think this is a sign that you are not designing maintainable systems in the first place. I work on reasonably complex systems where significant changes to a feature do not tend to cause huge structural changes like you describe, and where the vast majority of tests tend to remain unchanged and help detect regressions. It is entirely possible, with the right design. And in my experience, developing a comfort with TDD will help promote that kind of design. (Edit: I want to clarify that this is not meant in any kind of a judgmental way about your work or your talents, I'm sincerely trying to help identify issues so that you can hopefully become more productive and more confident in the work you produce.)

eyelidlessness 7y ago

> How do you deal with tests that end up self-referencing, testing only specific implementation details and adding no additional safety? Generally, by describing tests in plain english that express "user" intent (where "user" may be the dev at the call site, or may be an end user) rather than expressing things like return values or state changes. > In order to add or change a feature, you now have to write the code twice: Once to implement the change, and once to re-implement the change to assert that it does what it's supposed to. From this description, it doesn't sound like you're doing TDD. 1. Write a failing test (where "test" describes user intent). 2. Make that test pass with an implementation. 3. Make that implementation meet your expectations for code quality, appropriate separation of concerns, appropriate abstraction, DRY etc. (This is commonly referred to as red-green-refactor.) > And no safety has been added, because the logic itself can still be flawed. You've just written it twice instead of once. While you can always get the logic wrong, it's significantly less likely if what you're testing is intent. I really don't understand what you mean by "written it twice". > Generally I find tests that test the controllers, where multiple units of work are attached together, to be much more useful. I'm... honestly surprised by that. Those tests tend to be the most brittle, the least likely to cover all scenarios you'd like to test, the most error prone, and the hardest to understand and maintain.

jniedrauer 7y ago
> Those tests tend to be the most brittle While true, it's far more likely that they will help you during a refactor. If you use TDD and have a test for every function, you will more often than not throw entire functions and their tests away. A million tests that test immutable functions are all useless. But tests that fall in between TDD style test-per-function and integration tests, tend to be re-used post refactor, and give certainty that the refactor worked as expected.
- eyelidlessness 7y ago
  I think the size of a test shouldn't be defined by the size of the functionality you're testing, but rather a unit of "user intent", as expressed at whatever layer you're testing.