4 ms·

When you look at the TDD evangelists, all of them share something: they are all very good – probably even great – at design and refactoring. They see issues in

by dkarl 9y ago

When you look at the TDD evangelists, all of them share something: they are all very good – probably even great – at design and refactoring. They see issues in existing code and they know how to transform the code so it doesn’t have those issues, and specifically, they know how to separate concerns and reduce coupling. I think one of the selling points of TDD, and something I hoped for from TDD, was that causation went the other way, and writing tests would result in code being refactored to separate concerns and reduce coupling. Sadly, I've seen that it is possible to write code that is highly testable but is still a confused mess. What's more, TDD as promoted encourages people to confuse the two, resulting in testability being used as a reliable indicator of good design, which produces poor results because it's much easier to make code testable than it is to make it well-designed. I've also seen people mangle well-factored but untestable code in the process of writing tests, which can be a tragedy when dealing with a legacy codebase that was written with insufficient testing but is otherwise well-designed. A legacy codebase should always be approached with respect until you learn to inhabit the mental space of the people who wrote it (always an incomplete process yet very important), but TDD encourages people to treat untested code as crap and start randomly hacking it up as if it were impossible to make it worse. This unfortunate (and lazy) habit of treating testability as identical with good design is not a mistake that good TDD practitioners would make, but I think they did make a mistake in the understanding of their process. My guess is that when those people invested effort into refactoring their code for testability, they were improving the design at the same time, as a side effect of the time and attention invested combined with their natural tendency to recognize and value good design. They misunderstood that process and gave too much credit to the pursuit of testability as naturally leading to better design. I do think the idea of TDD is not entirely bankrupt, because the value of writing tests is more than just the value of having the tests afterwards, but I think its value is overblown, and people who believe in the magical effect of TDD end up having blind confidence in the quality of their code.

agentultra 9y ago
> I've also seen people mangle well-factored but untestable code in the process of writing tests, which can be a tragedy when dealing with a legacy codebase that was written with insufficient testing but is otherwise well-designed. Have you read Michael Feathers' Working Effectively with Legacy Code? [0] In his definition of legacy code it is any such code that has no test coverage. It's a black box. There are errors in it somewhere. It works for some inputs. However you cannot quantify either of those things just by "inhabiting the mind of the original developers." The only way to work effectively with that code base in order to extend it, maintain it, or modify it is to bring it under test. This is far more difficult than it sounds with legacy code than with greenfield TDD for the aforementioned reasons: there are unquantified errors and underspecified behaviours. You can't possibly do it in one sweeping effort and so the strategy is to accept that tests are useful and to add them with each change, first, before making that change and using the test to prove the change is correct. Slowly, over time, your legacy code base surfaces little islands of well tested code. You have to be deliberate and careful. You have to think about what you're doing. This is a much different experience than writing greenfield code. TDD is effortless and drives you towards the answer in this case. [0] https://www.amazon.com/Working-Effectively-Legacy-Michael-Feathers/dp/0131177052 https://www.amazon.com/Working-Effectively-Legacy-Michael-Fe...
- dkarl 9y ago
  Yeah, I've read it, I know the definition, and I understand the intention behind adding tests to legacy code. Unfortunately, TDD (meaning TDD as I've encountered it in print and in practice) encourages people to think that "no tests" is tantamount to "no value to preserve" and therefore "no risk of harm from refactoring." Maybe that's an exaggeration, but certainly some TDD practitioners think that they can't possibly harm a codebase by adding tests. Unfortunately, testing requires refactoring, refactoring is redesigning, and if you don't understand the design you're modifying, your changes can make the code less understandable, not more. Tests added after the fact impose the test-writer's understanding of how the system works, which results in chaos if their understanding isn't compatible with the understanding embedded in the existing design. Also, in spite of that definition, there's a lot more to "legacy" than not having tests. Not having access to the original designer and not having access to the the requirements or domain understanding that influenced development are important handicaps of legacy code that are entirely separate from tests. Certainly tests added during development can capture some of this knowledge, but adding tests after the fact does not automatically recreate it. These are both examples of the danger of trying to elevate one aspect of software development to a primary and sufficient role. "Take care of X and everything else will work out" has no known solution for software development, and any methodology is harmful to the extent that it encourages people to think that way.
- ericgunnerson 9y ago
  Original author here... Your writeup mirrors my personal experience, which was why I was such an advocate for so long. But it doesn't mirror my team experience at all. I bought Feather's book when it first came out and liked it so much that I bought copies for a whole team out of my own pocket. And I talked about and taught the techniques to the team members. I had high hopes, and a good team, but the results that we got were not what I hoped for. The result we got was a lot of small tested functions throughout the codebase ("sprouts" in Feather's nomenclature) - which I expected - but I didn't see any evidence of coalescing of those tests into something better - and by themselves, the sprouts made the codebase worse. I ran into a number of cases where I would pair with developers who were looking to do something better than sprout - so they could fix a whole class or area to be better - and the best answer I could give them is, "give me a morning and I can probably come up with a good solution", which doesn't really help. Some code is just aggressively hard to test. And I should mention that this was pretty much the perfect environment to try this; I had a team without shipping pressure and a mandate from above to write better unit tests and sufficient time to train and pair with developers. The approach we settled on was different. We spent our time looking for "targets of opportunity"; the problematic classes or groups of classes that were leading to real issues - bug farms, hard to modify, that sort of thing - and we took a targeted approach to replace them. The replacements were generally written using TDD.

by dkarl 9y ago

agentultra 9y ago

> I've also seen people mangle well-factored but untestable code in the process of writing tests, which can be a tragedy when dealing with a legacy codebase that was written with insufficient testing but is otherwise well-designed. Have you read Michael Feathers' Working Effectively with Legacy Code? [0] In his definition of legacy code it is any such code that has no test coverage. It's a black box. There are errors in it somewhere. It works for some inputs. However you cannot quantify either of those things just by "inhabiting the mind of the original developers." The only way to work effectively with that code base in order to extend it, maintain it, or modify it is to bring it under test. This is far more difficult than it sounds with legacy code than with greenfield TDD for the aforementioned reasons: there are unquantified errors and underspecified behaviours. You can't possibly do it in one sweeping effort and so the strategy is to accept that tests are useful and to add them with each change, first, before making that change and using the test to prove the change is correct. Slowly, over time, your legacy code base surfaces little islands of well tested code. You have to be deliberate and careful. You have to think about what you're doing. This is a much different experience than writing greenfield code. TDD is effortless and drives you towards the answer in this case. [0] https://www.amazon.com/Working-Effectively-Legacy-Michael-Feathers/dp/0131177052 https://www.amazon.com/Working-Effectively-Legacy-Michael-Fe...

dkarl 9y ago
Yeah, I've read it, I know the definition, and I understand the intention behind adding tests to legacy code. Unfortunately, TDD (meaning TDD as I've encountered it in print and in practice) encourages people to think that "no tests" is tantamount to "no value to preserve" and therefore "no risk of harm from refactoring." Maybe that's an exaggeration, but certainly some TDD practitioners think that they can't possibly harm a codebase by adding tests. Unfortunately, testing requires refactoring, refactoring is redesigning, and if you don't understand the design you're modifying, your changes can make the code less understandable, not more. Tests added after the fact impose the test-writer's understanding of how the system works, which results in chaos if their understanding isn't compatible with the understanding embedded in the existing design. Also, in spite of that definition, there's a lot more to "legacy" than not having tests. Not having access to the original designer and not having access to the the requirements or domain understanding that influenced development are important handicaps of legacy code that are entirely separate from tests. Certainly tests added during development can capture some of this knowledge, but adding tests after the fact does not automatically recreate it. These are both examples of the danger of trying to elevate one aspect of software development to a primary and sufficient role. "Take care of X and everything else will work out" has no known solution for software development, and any methodology is harmful to the extent that it encourages people to think that way.
ericgunnerson 9y ago
Original author here... Your writeup mirrors my personal experience, which was why I was such an advocate for so long. But it doesn't mirror my team experience at all. I bought Feather's book when it first came out and liked it so much that I bought copies for a whole team out of my own pocket. And I talked about and taught the techniques to the team members. I had high hopes, and a good team, but the results that we got were not what I hoped for. The result we got was a lot of small tested functions throughout the codebase ("sprouts" in Feather's nomenclature) - which I expected - but I didn't see any evidence of coalescing of those tests into something better - and by themselves, the sprouts made the codebase worse. I ran into a number of cases where I would pair with developers who were looking to do something better than sprout - so they could fix a whole class or area to be better - and the best answer I could give them is, "give me a morning and I can probably come up with a good solution", which doesn't really help. Some code is just aggressively hard to test. And I should mention that this was pretty much the perfect environment to try this; I had a team without shipping pressure and a mandate from above to write better unit tests and sufficient time to train and pair with developers. The approach we settled on was different. We spent our time looking for "targets of opportunity"; the problematic classes or groups of classes that were leading to real issues - bug farms, hard to modify, that sort of thing - and we took a targeted approach to replace them. The replacements were generally written using TDD.