5 ms·

TDD and OOP as dogmas are very bad, for different reasons. TDD encourages you to write many times the number of lines of code for your tests as the code you're

TDD and OOP as dogmas are very bad, for different reasons. TDD encourages you to write many times the number of lines of code for your tests as the code you're testing, often 10X or more. So when you inevitably decide "there's a much better way to do this" (which happens to me 100% of the time) then you're not only changing your code, you're changing all of those tests. That's a lot of weight that you now have to deal with. Most of the time, that's enough weight that the better design simply doesn't happen and becomes yet another chunk of tech debt that prevents certain things from happening in the future. I've seen that happen, and it's given me a very bad taste in my mouth for "TDD" because "TDD inertia" is the reason that better designs aren't implemented. If that piece of software lives for a while, and grows in scope, someone is going to have to deal with that, and eventually make the architectural change anyway. By all means, test your code, of course. If you can find a way to easily generate test cases for an arbitrary code base, by all means, do that, because the test cases are no longer influencing your decision to change how your application works. Otherwise, being "test-driven" is bad, IMHO. Software development is never as simple as the various dogmas would lead you to believe.

CharlesW 5y ago
> So when you inevitably decide "there's a much better way to do this" (which happens to me 100% of the time) then you're not only changing your code, you're changing all of those tests. Isn't the point of TDD that you can change implementation at will, safe in the knowledge that the tests will help guarantee that you're not inadvertently changing behavior?
- cryvate1284 5y ago
  This depends on what you're doing. For example, if you are changing an intermediary (abstraction) layer, then the (unit) tests (TDD or not) for those will have to change and the "guarantee that you're not inadvertently changing behaviour" is kinda moot. If you do not have TDD, I guess the reasoning is that either you do not have tests for this (probably bad) and if you do, less of them (and so more easily changed). Not sure it's an argument against TDD, but I guess if management/the programmer do not know about sunk cost fallacy, it might make them hold on to bad abstractions/layers.
  - CharlesW 5y ago
    Thanks, that's helpful!
- ema 5y ago
  Often the better way to do something is achieved by changing how two components interface with each other. in this case you have to change the tests that test this interface.
- bogdanoff_2 5y ago
  Unless what you're testing has a very clear interface boundary (like a pure function whose behavior is unlikely changed, or the program's end-to-end behavior), there's no clear distinguish between behavior and implementation. Most of the time you have classes that interact with one another. If you test each class individually, and you decide to refactor what classes do what, you'll need to rewrite a lot of tests, even though the higher level behavior should not change.
- twh270 5y ago
  Usually, yes -- your tests should be decoupled from the implementation. Usually however, what I see (in Java and Spring Boot land) is test methods which use mocks that are set up to expect certain method calls, e.g. when(accountService.getAccount(id)).thenReturn(account). Typically there are anywhere from six to 20 lines of this stuff per test (and, it's rarely de-duplicated into a separate method). So as soon as the contract to getAccount() changes, a bunch of tests need to change. Second, it's common to believe that the "unit" in a "unit test" is either a class or (less often) a method. Consequently, people write test classes for every single class of a group of classes that are acting in collaboration, when what they really ought to do is write tests against the single class supplying the public interface, varying input to that class as necessary to achieve good confidence.
kllrnohj 5y ago
I think kovarex is avoiding your TDD concerns by rejecting the notion that everything must be covered by minimal-scope unit tests. This means that there's much fewer (if any) mocks that need updating when the implementation changes, and fewer tests should end up needing to be touched as a result as well. See specifically the "Fig. 5 - Test dependencies" section of the post. Personally I think mocks are far over-used in tests, and I much prefer the solution kovarex outlines. I think dealing with mocks are the bigger source of test-update friction here than the tests themselves. I was already layering my tests instead of using mocks. As in having no clear line between "unit" and "integration" tests, everything just uses the "real" implementations of things and tests just get naturally more complex the higher up the stack it's testing. The idea of sorting by dependency depth is a cool idea I hadn't ever considered, though, I'll be borrowing that idea.
imiric 5y ago
> TDD encourages you to write many times the number of lines of code for your tests as the code you're testing, often 10X or more. If that happens I'd say you're doing TDD too early. TDD can be very useful, but early on in the process of transferring a design to code, all your interfaces are highly unstable and there's a lot of experimentation, so sticking strictly to TDD would naturally be frustrating. I would start a bit later when at least some of the API has stabilized and you don't have to do major changes. TDD helps with creating user friendly APIs, since you experience it from the user's perspective. And it forces you to actually write testable code and not incur technical debt that's very costly to remove later (having to refactor code to make it testable). > Most of the time, that's enough weight that the better design simply doesn't happen and becomes yet another chunk of tech debt that prevents certain things from happening in the future. This reads like you're saying that tests themselves are technical debt...? Because you're going to run into this issue (or should run into it) regardless if you use TDD or not. Eventually you'll want to refactor parts of your codebase and, sure, you'll have to change some of your, hopefully mostly, unit tests. So in that sense you can say that doing TDD early creates a lot of technical debt that needs to be resolved quickly, but like I said above, that doesn't have to be the case. TDD can be a dogma just like any practice (Agile is my favorite), but it doesn't mean that it's not useful if used correctly. Kudos to the Factorio team for adopting it, which I think is rare in the gaming industry. The idea alone of testing the complexities of a video game is mind boggling to me as a web developer. Especially in an industry where it's popular to hype and sell broken products with the promise of patches and DLC.
- naikrovek 5y ago
  does TDD not dictate that you write tests first, before you write the code under test? then you write only enough code to make the test pass, as I recall. yeah TDD is a mess
  - imiric 5y ago
    Why does a practice have to "dictate" anything? TDD is a workflow suggestion, not gospel that must be strictly followed. Writing and maintaining a test before any experiments have been done with the design would indeed be frustrating and lead to a lot of rewriting. If you first explore the design and allow it to settle into a usable interface and then write the test for it as assurance that it's stable, you'd have the benefit of being able to safely refactor the feature and quickly add more tests for it. As long as the deliverable includes tests with decent coverage, who cares whether you wrote a test first without writing a single line of the implementation?
pault 5y ago
I always thought of TDD as a way to exercise your code while writing it, not a set in stone spec of your product. I consider unit tests disposable and basically noise; they're only useful when you're writing code. They are by nature a test of your implementation, not your specification. I usually throw away a lot of them after I'm done writing, and if I refactor I just delete the ones that are failing because I'm writing new ones as I go. There are places at boundaries where more stable unit tests can go, but even there a change to the interface is going to break all the tests anyway. The better way to test your specification and behavior, IMO, is to use comprehensive E2E tests. The traditional test pyramid is upside down. If you have an E2E test for every user story in your spec, you can develop with confidence that your new code will not disrupt your users' activity. Unit tests are cattle, E2E tests are pets.

CharlesW 5y ago

> So when you inevitably decide "there's a much better way to do this" (which happens to me 100% of the time) then you're not only changing your code, you're changing all of those tests. Isn't the point of TDD that you can change implementation at will, safe in the knowledge that the tests will help guarantee that you're not inadvertently changing behavior?

cryvate1284 5y ago
This depends on what you're doing. For example, if you are changing an intermediary (abstraction) layer, then the (unit) tests (TDD or not) for those will have to change and the "guarantee that you're not inadvertently changing behaviour" is kinda moot. If you do not have TDD, I guess the reasoning is that either you do not have tests for this (probably bad) and if you do, less of them (and so more easily changed). Not sure it's an argument against TDD, but I guess if management/the programmer do not know about sunk cost fallacy, it might make them hold on to bad abstractions/layers.
- CharlesW 5y ago
  Thanks, that's helpful!
ema 5y ago
Often the better way to do something is achieved by changing how two components interface with each other. in this case you have to change the tests that test this interface.
bogdanoff_2 5y ago
Unless what you're testing has a very clear interface boundary (like a pure function whose behavior is unlikely changed, or the program's end-to-end behavior), there's no clear distinguish between behavior and implementation. Most of the time you have classes that interact with one another. If you test each class individually, and you decide to refactor what classes do what, you'll need to rewrite a lot of tests, even though the higher level behavior should not change.
twh270 5y ago
Usually, yes -- your tests should be decoupled from the implementation. Usually however, what I see (in Java and Spring Boot land) is test methods which use mocks that are set up to expect certain method calls, e.g. when(accountService.getAccount(id)).thenReturn(account). Typically there are anywhere from six to 20 lines of this stuff per test (and, it's rarely de-duplicated into a separate method). So as soon as the contract to getAccount() changes, a bunch of tests need to change. Second, it's common to believe that the "unit" in a "unit test" is either a class or (less often) a method. Consequently, people write test classes for every single class of a group of classes that are acting in collaboration, when what they really ought to do is write tests against the single class supplying the public interface, varying input to that class as necessary to achieve good confidence.

kllrnohj 5y ago

I think kovarex is avoiding your TDD concerns by rejecting the notion that everything must be covered by minimal-scope unit tests. This means that there's much fewer (if any) mocks that need updating when the implementation changes, and fewer tests should end up needing to be touched as a result as well. See specifically the "Fig. 5 - Test dependencies" section of the post. Personally I think mocks are far over-used in tests, and I much prefer the solution kovarex outlines. I think dealing with mocks are the bigger source of test-update friction here than the tests themselves. I was already layering my tests instead of using mocks. As in having no clear line between "unit" and "integration" tests, everything just uses the "real" implementations of things and tests just get naturally more complex the higher up the stack it's testing. The idea of sorting by dependency depth is a cool idea I hadn't ever considered, though, I'll be borrowing that idea.

imiric 5y ago

> TDD encourages you to write many times the number of lines of code for your tests as the code you're testing, often 10X or more. If that happens I'd say you're doing TDD too early. TDD can be very useful, but early on in the process of transferring a design to code, all your interfaces are highly unstable and there's a lot of experimentation, so sticking strictly to TDD would naturally be frustrating. I would start a bit later when at least some of the API has stabilized and you don't have to do major changes. TDD helps with creating user friendly APIs, since you experience it from the user's perspective. And it forces you to actually write testable code and not incur technical debt that's very costly to remove later (having to refactor code to make it testable). > Most of the time, that's enough weight that the better design simply doesn't happen and becomes yet another chunk of tech debt that prevents certain things from happening in the future. This reads like you're saying that tests themselves are technical debt...? Because you're going to run into this issue (or should run into it) regardless if you use TDD or not. Eventually you'll want to refactor parts of your codebase and, sure, you'll have to change some of your, hopefully mostly, unit tests. So in that sense you can say that doing TDD early creates a lot of technical debt that needs to be resolved quickly, but like I said above, that doesn't have to be the case. TDD can be a dogma just like any practice (Agile is my favorite), but it doesn't mean that it's not useful if used correctly. Kudos to the Factorio team for adopting it, which I think is rare in the gaming industry. The idea alone of testing the complexities of a video game is mind boggling to me as a web developer. Especially in an industry where it's popular to hype and sell broken products with the promise of patches and DLC.

naikrovek 5y ago
does TDD not dictate that you write tests first, before you write the code under test? then you write only enough code to make the test pass, as I recall. yeah TDD is a mess
- imiric 5y ago
  Why does a practice have to "dictate" anything? TDD is a workflow suggestion, not gospel that must be strictly followed. Writing and maintaining a test before any experiments have been done with the design would indeed be frustrating and lead to a lot of rewriting. If you first explore the design and allow it to settle into a usable interface and then write the test for it as assurance that it's stable, you'd have the benefit of being able to safely refactor the feature and quickly add more tests for it. As long as the deliverable includes tests with decent coverage, who cares whether you wrote a test first without writing a single line of the implementation?

pault 5y ago

I always thought of TDD as a way to exercise your code while writing it, not a set in stone spec of your product. I consider unit tests disposable and basically noise; they're only useful when you're writing code. They are by nature a test of your implementation, not your specification. I usually throw away a lot of them after I'm done writing, and if I refactor I just delete the ones that are failing because I'm writing new ones as I go. There are places at boundaries where more stable unit tests can go, but even there a change to the interface is going to break all the tests anyway. The better way to test your specification and behavior, IMO, is to use comprehensive E2E tests. The traditional test pyramid is upside down. If you have an E2E test for every user story in your spec, you can develop with confidence that your new code will not disrupt your users' activity. Unit tests are cattle, E2E tests are pets.