5 ms·
When people say "testable code" I always wince. Unit tests fucking suck at testing most code. Really suck. They're ok (ok, still not great) for things like pars
by pydry 6y ago
When people say "testable code" I always wince. Unit tests fucking suck at testing most code. Really suck. They're ok (ok, still not great) for things like parsers and algorithms... which isn't most code.
But, rather than improving test tools to handle more types of code, developers say that they want to completely restructure the code to make it more amenable to the fact that unit tests suck. Unit tests can't handle database? You need "Testable code"! No...
Developers code to a spec, but unit tests suck at encoding a spec except in the few cases you alluded to simply because most specs are not easily, clearly encoded in the form of your turing complete programming language with its mediocre tooling.
The practical upshot of this theoretical brain damage is that when people write unit tests on code where it isn't a suitable tool it tends to be an expensive waste of time. The tests cost time to build, time to maintain and when they fail it means... "oh, you changed some code". Thanks, test.
And then religious unit testing people always argue that it wasn't the tool that was at fault it was you.
Working on projects with excellent integration test tooling really opened my eyes to the possibilities of another world - one without unit tests.
- jchw 6y agoI don’t have much to say beyond what I already did re “testable” code; forcing code to be “testable” can be bad, but in reality making code more testable probably mostly amounts to breaking it down into simpler, functionally pure bits. Not everything unit tests terribly well. I haven’t seen it work very well for things like React or Angular components. It does work well for small bits of apps. Like for example, lets say you have a component where a good deal of what it does is extracts information out of a URL. You might have tests that exercise the entire UI, entering URLs and checking the HTML output. The “more testable” version of this, imo, would be separating the URL parsing and extraction bit to a single free-standing routine that outputs some data given an input URL. You can then table test that bit. Then testing that this ties into the UI correctly could be done in the integration or end to end testing. In case of table based unit testing, I think it often works great, as it can act as a running log of regressions and newly discovered edge cases that can even serve as a sort of document of expectations for other developers, and while it cannot be used to test all sorts of code, it has wide applicability and you can see it in webapps, Go servers, the Wine and libinput sources, etc. It’s easy to get stuck on a single strategy to rule them all, but I think that often is a bit presumptuous and maybe dogmatic. Tests are a toolbox. Not every problem is a nail.
- pydry 6y ago>in reality making code more testable probably mostly amounts to breaking it down into simpler, functionally pure bits. I know. The problem is: * This process often introduces bugs. How are you going to catch those bugs? Not with your tests, you're changing this code precisely so you can write tests. It's a catch 22. * Sometimes people do this only to discover that the simpler "functionally" pure code is pointless to test because it's so trivially simple. Somebody literally did that today on the code base I work on. The code as a whole still has bugs but those tests won't ever catch one. They'll just break when the code changes. Plus that "refactoring" probably introduced bugs. This I think is what the concept of "unit test induced design damage" was getting at. This isn't a problem with tests as a whole. Or TDD. It is partly a problem with people who use the terms "unit test" and "test" interchangeably (this engenders entirely the wrong kind of thinking). It's mostly a problem with unit testing as a concept (i.e. not the specific frameworks themselves). Having nice clean code interfaces is also often conflated with unit testing - this is a mistake. One does not necessarily lead to the other.
- Tainnor 6y agoMichael Feather's "Working Effectively with Legacy Code" goes into some detail on this. When you have bad, untested code, you don't start refactoring. You start writing high level "characterisation tests", then you refactor. After that you can still write the unit tests for the better components.
- pydry 6y agoI've followed this process a few times and then noticed that the last step didn't really add much value. It felt good at the time doing it because that's what I was "supposed" to do. I'd achieved the supposed "testable code Nirvana" and... meh. The first step was life (or at least, career) changing though. Bringing a piece of shit code base under control with integration tests was a process that blew my mind. That's what led me to start questioning the efficacy of jamming architectural changes into code in order to sacrifice at the altar of the unit testing gods and that maybe, just maybe, unit tests' steep demands and limited value means that they suck.
- exdsq 6y agoYou need a combination of unit tests and integration tests. Even trivial tests can help prevent someone ‘fixing’ your code in a refactor and breaking it, by defining the properties of what the code should do through tests. But it depends on the project - sometimes it just doesn’t matter, sometimes you need more formal V&V.
- pydry 6y agoYou need to think beyond the concept of unit tests and integration tests. The future will have higher level and lower level executable specifications. The future won't have "unit tests" and "integration tests".
- exdsq 6y agoI work in a formal methods environment with proof checkers, property based testing, Isabelle/Coq/TLA+, and we still have unit tests and integration tests as concepts. You need a common language and these work well.
- rocqua 6y agoThe distinction is easy. A unit test test whether a single unit works as expected. Generally, a failed unit test comes with a good pointer to what is wrong, and failed unit tests are rare. Meanwhile integration tests test whether multiple units are integrated properly to produce expected results. When an integration test fails, you don't really know what is wrong. Instead you look at or write unit tests for the smaller units to see whether they fail, or whether the way you tied them together has a flaw.
- yoden 6y agoWhat do you see as good integration test tooling?
- pydry 6y agoSomething where the integration tests: * Are clear and easy to read and form a specification of sorts. * Which are cheap to build. * Where the interactions the code has to the "outside world" are easy and cheap to mock/test/keep under control - e. e.g. time, database, browsers. * Has good debugging tools such that it's easy to track down the source of the bugs.
- leafboi 6y agoYou're talking about things like dependency injection. I'm not a huge fan of testing but "testable code" when written in a certain way (without dependency injection) actually has many external benefits that don't have to do with testing per se. Really the way you need to structure your program is to divide functions in your code between things that can be unit tested and can't (IO calls). This can easily be done without dependency injection which is likely what you're complaining about. For example Don't do this: #unit testable with mock (Bad!) function add_one(key: str, database: Database): return database.get(key) + 1 Or.. even worse: #Unit testable with mock (Bad!) class Adder Adder(database) -> None { this.database = database } add_one(key: str) -> int { return this.database.get(key) + 1 } Do this instead: #IO function, Not unit testable def get_database_value(key, database) -> None: return database.get(key) #Unit testable without mock! Good! def add_one(value: int) -> int: return value + 1 #IO function, not unit testable. def composition(key: str) -> int: return add_one(get_database_value("name"))
- marcosdumay 6y agoUnit tests are great, for testing unities. What self-contained unities are there on your code? If you are doing low level system programming, I bet there are a lot of them. If you are doing high-level CRUD, I bet there is none, all of them you import from third parties. Writing unit tests for non-self-contained code is crazy, and leads to all those problems people identify. In my experience, the problem is that the most vocal evangelist believers of unit test are all on places that work on the high level stuff.