6 ms·
I've discovered that adding unit tests to an existing code base doesn't work very well, as each piece of code tends to have too many interdependencies with othe
by WalterBright 4y ago
I've discovered that adding unit tests to an existing code base doesn't work very well, as each piece of code tends to have too many interdependencies with others. The code has to be organized from the ground up to be compatible with unit tests.
- paulddraper 4y agoI find unit tests to be a good fit for 10% of code. Most of the time, bugs are in the glue.
- beebmam 4y agoI find that most of my development time in my career is spent getting that glue in place! Seems like we need a better way to glue things together and make sure that glue doesn't make a mess of things. Maybe AI can help with this
- jaggederest 4y agoThere are tests that help ensure that things are glued correctly. Functional testing is useful in this regard, and end-to-end (sometimes called integration testing*) is extremely useful in making sure critical parts of the system work. For example, I always try to write a "log in" test, and for systems that take payment, a "payment acceptance" test. Even if other parts of the system have bugs, I want to be able to have existing clients log in, and new clients sign up, no matter what. * though I think integration testing is really "anything that tests more than one system" without necessarily being fully end to end
- beebmam 4y agoNot enough companies use these end-to-end tests you describe!
- jaggederest 4y agoI put them in place everywhere I go, so I'm trying my best! I think in a lot of cases the value is difficult to see unless you maintain them as a priority - they can get out of date and start randomly failing, and that's when they go "the way of the dodo" in most organizations, even though a few hours a month of engineering time deliver significant, useful results for your investment.
- saurik 4y agoWhich is sad, because well chosen / planned integration tests also can have high code coverage, but no matter how good your unit tests are they will never replace the need for an integration test. Imagine if someone test every component of a car but didn't do a routine road test, right?
- bcrosby95 4y agoIf you have layers that look like: view -> controller -> application services -> model/db, I focus most of my automated tests on everything at the application services. Critically, I don't mock our db, instead I focus on making setup and teardown fast. It seems to get me the most bang for my buck. Then I have a few end-to-end tests for more critical flows.
- WalterSear 4y agoThat's what stricter types are for. And BDD style feature tests, which I find vastly more useful than either Unit tests or E2E tests. Unit tests have their place - in maybe 10% of the code :) E2E tests are also useful for a smaller purpose - mostly as happy-path sanity checks rather than in guiding development. IMHO.
- jaggederest 4y agoThere are lots of good techniques for adding unit tests to an existing codebase that does not have unit tests without reorganizing everything from the ground up. I like Michael Feathers book, "Working Effectively with Legacy Code" - his definition of legacy code is simply code that does not have tests, so the whole book is more or less about how to solve this problem. One technique he references that I find very effective is building "firewalls" between different parts of the codebase, and then unit testing those firewalls thoroughly. This means that you can isolate a bug to one part of the system or another, and drastically reduces the surface area to look at when trying to find why there is a bug or why a test is failing. Edit: In fact, I've seen places in your D code that use similar techniques already - maybe just giving them a name makes a big difference?
- WalterBright 4y agoI haven't read his work, but the firewalls sound like encapsulating things to reduce interdependencies, which indeed is what makes unittests feasible. The data structures that go in and out of the modules then get replaced by "mock" data structures.
- rad_gruchalski 4y agoFor me this usually means that the code needs to be refactored to make chunks easier to test, or an integration/e2e test is needed, sometimes with the world running in containers (extreme cases when there’s no access to dependency source).
- WalterSear 4y agoIn my personal, subjective and very non-prescritive experience, unit tests aren't very useful for much. Behavioral Driven Development (i.e. Cucumber tests) at the module level are what I start with when attempting to corral musty code. I totally agree that, even then, wrangling ungroomed interdependencies are definitely the biggest headache. But I have yet to adopt this approach and not think to myself, "why didn't I start this sooner?"
- bmitc 4y ago> In my personal, subjective and very non-prescritive experience, unit tests aren't very useful for much. Why is that? In more functional oriented languages, they are great because they allow you to verify behavior of functions. Then in the broader application, you're just composing pieces that have already been validated, and the compositions can also be tested. If you're able to incorporate property-based testing, then that's even better.
- WalterSear 4y ago* Because I care about function, not implementation * Writing tests has a cost, so I want to right as few as possible while ensuring functionality. * rewriting tests has a cost. When I start writing, I know what functionality I want, but I may iterate significantly on implementation. This makes unit tests both an impediment, and providing me no safety to perform the refactor/reimplementation. * the module, or it's equivalent structure in your language, is the optimal tradeoff between standalone functionality. Ideally, a module encompasses a small enough behaviour to fit in one's head. IMHO, an ideal single function should be small enough that it's behavior is readably obvious. * module inputs and dependencies are easier to instrument in tests. I/O and hardware API dependencies can be injected/overridden. On the other hand, private variables and internal instantiations can be complex and hard to mock appropriately when unit testing, unless you are able to structure absolutely everything as a pure function - IMHO a platonic ideal, particularly in the workplace. * Functional tests are easier for other coders (and even non-coders, in regards to feature files) to understand. "I wrote 100 unit tests" provides less information regarding actual functionality and productivity than "this feature's scenarios are all passing now." The originators of unit tests and extreme programming explicitly stated that unit tests are not to be checked into source control, and deleted instead. They felt that unit tests would overly constrain subsequent programmers, and that higher level suites were more useful at providing refactoring safety. The only time to submit a smaller sized test is as part of a bug fix, where one would write a failing test, fix the code and check it in. Fwiw, while I agree with them on this, I'm don't want to come across as appealing to authority. A lot of Agile/eXtreme/Scrum ideas reflect a simpler and very different time in coding, when code was, by necessity simpler and easier to reason about, and the culture of computers gave engineers much more power to push back on management demands and prioritize code quality. And some of the ideas are just hokey - like eXtreme programmings dislike for branches - encouraging all developers on a team to land their code on 'main' at the end of every day. Besides, the very first Agile project -that birthed the movement - went over time and over budget :)