4 ms·

> I've also seen people mangle well-factored but untestable code in the process of writing tests, which can be a tragedy when dealing with a legacy codebase tha

> I've also seen people mangle well-factored but untestable code in the process of writing tests, which can be a tragedy when dealing with a legacy codebase that was written with insufficient testing but is otherwise well-designed. Have you read Michael Feathers' Working Effectively with Legacy Code? [0] In his definition of legacy code it is any such code that has no test coverage. It's a black box. There are errors in it somewhere. It works for some inputs. However you cannot quantify either of those things just by "inhabiting the mind of the original developers." The only way to work effectively with that code base in order to extend it, maintain it, or modify it is to bring it under test. This is far more difficult than it sounds with legacy code than with greenfield TDD for the aforementioned reasons: there are unquantified errors and underspecified behaviours. You can't possibly do it in one sweeping effort and so the strategy is to accept that tests are useful and to add them with each change, first, before making that change and using the test to prove the change is correct. Slowly, over time, your legacy code base surfaces little islands of well tested code. You have to be deliberate and careful. You have to think about what you're doing. This is a much different experience than writing greenfield code. TDD is effortless and drives you towards the answer in this case. [0] https://www.amazon.com/Working-Effectively-Legacy-Michael-Feathers/dp/0131177052 https://www.amazon.com/Working-Effectively-Legacy-Michael-Fe...

dkarl 9y ago
Yeah, I've read it, I know the definition, and I understand the intention behind adding tests to legacy code. Unfortunately, TDD (meaning TDD as I've encountered it in print and in practice) encourages people to think that "no tests" is tantamount to "no value to preserve" and therefore "no risk of harm from refactoring." Maybe that's an exaggeration, but certainly some TDD practitioners think that they can't possibly harm a codebase by adding tests. Unfortunately, testing requires refactoring, refactoring is redesigning, and if you don't understand the design you're modifying, your changes can make the code less understandable, not more. Tests added after the fact impose the test-writer's understanding of how the system works, which results in chaos if their understanding isn't compatible with the understanding embedded in the existing design. Also, in spite of that definition, there's a lot more to "legacy" than not having tests. Not having access to the original designer and not having access to the the requirements or domain understanding that influenced development are important handicaps of legacy code that are entirely separate from tests. Certainly tests added during development can capture some of this knowledge, but adding tests after the fact does not automatically recreate it. These are both examples of the danger of trying to elevate one aspect of software development to a primary and sufficient role. "Take care of X and everything else will work out" has no known solution for software development, and any methodology is harmful to the extent that it encourages people to think that way.
ericgunnerson 9y ago
Original author here... Your writeup mirrors my personal experience, which was why I was such an advocate for so long. But it doesn't mirror my team experience at all. I bought Feather's book when it first came out and liked it so much that I bought copies for a whole team out of my own pocket. And I talked about and taught the techniques to the team members. I had high hopes, and a good team, but the results that we got were not what I hoped for. The result we got was a lot of small tested functions throughout the codebase ("sprouts" in Feather's nomenclature) - which I expected - but I didn't see any evidence of coalescing of those tests into something better - and by themselves, the sprouts made the codebase worse. I ran into a number of cases where I would pair with developers who were looking to do something better than sprout - so they could fix a whole class or area to be better - and the best answer I could give them is, "give me a morning and I can probably come up with a good solution", which doesn't really help. Some code is just aggressively hard to test. And I should mention that this was pretty much the perfect environment to try this; I had a team without shipping pressure and a mandate from above to write better unit tests and sufficient time to train and pair with developers. The approach we settled on was different. We spent our time looking for "targets of opportunity"; the problematic classes or groups of classes that were leading to real issues - bug farms, hard to modify, that sort of thing - and we took a targeted approach to replace them. The replacements were generally written using TDD.

dkarl 9y ago

Yeah, I've read it, I know the definition, and I understand the intention behind adding tests to legacy code. Unfortunately, TDD (meaning TDD as I've encountered it in print and in practice) encourages people to think that "no tests" is tantamount to "no value to preserve" and therefore "no risk of harm from refactoring." Maybe that's an exaggeration, but certainly some TDD practitioners think that they can't possibly harm a codebase by adding tests. Unfortunately, testing requires refactoring, refactoring is redesigning, and if you don't understand the design you're modifying, your changes can make the code less understandable, not more. Tests added after the fact impose the test-writer's understanding of how the system works, which results in chaos if their understanding isn't compatible with the understanding embedded in the existing design. Also, in spite of that definition, there's a lot more to "legacy" than not having tests. Not having access to the original designer and not having access to the the requirements or domain understanding that influenced development are important handicaps of legacy code that are entirely separate from tests. Certainly tests added during development can capture some of this knowledge, but adding tests after the fact does not automatically recreate it. These are both examples of the danger of trying to elevate one aspect of software development to a primary and sufficient role. "Take care of X and everything else will work out" has no known solution for software development, and any methodology is harmful to the extent that it encourages people to think that way.

ericgunnerson 9y ago

Original author here... Your writeup mirrors my personal experience, which was why I was such an advocate for so long. But it doesn't mirror my team experience at all. I bought Feather's book when it first came out and liked it so much that I bought copies for a whole team out of my own pocket. And I talked about and taught the techniques to the team members. I had high hopes, and a good team, but the results that we got were not what I hoped for. The result we got was a lot of small tested functions throughout the codebase ("sprouts" in Feather's nomenclature) - which I expected - but I didn't see any evidence of coalescing of those tests into something better - and by themselves, the sprouts made the codebase worse. I ran into a number of cases where I would pair with developers who were looking to do something better than sprout - so they could fix a whole class or area to be better - and the best answer I could give them is, "give me a morning and I can probably come up with a good solution", which doesn't really help. Some code is just aggressively hard to test. And I should mention that this was pretty much the perfect environment to try this; I had a team without shipping pressure and a mandate from above to write better unit tests and sufficient time to train and pair with developers. The approach we settled on was different. We spent our time looking for "targets of opportunity"; the problematic classes or groups of classes that were leading to real issues - bug farms, hard to modify, that sort of thing - and we took a targeted approach to replace them. The replacements were generally written using TDD.