4 ms·
Very, very few people know how to make cheap tests. I have one guy who writes fixtures that couples all of his tests. When I need to add functionality, first I
by hinkley 6y ago
Very, very few people know how to make cheap tests.
I have one guy who writes fixtures that couples all of his tests. When I need to add functionality, first I have to replace his fixtures with basic stubs, just so I can add another bit of functionality.
And we have two people who copy his coding style. Probably to fit in.
I think, based on my own experiences and working with people learning to write tests, that 'simple' tests often feel more like the pejorative meaning of the word. They get uncomfortable, like you're debasing yourself to write such boring code. Surely I can write more interesting tests.
Yes, you can write more interesting tests if your goal is for nobody to ever touch your code. Mine, mine, mine. The best tests, I know where my new code went wrong simply by reading the test output (not the test). If I refactor I should be able to maintain that coverage even if the order or structure of things gets completely switched around.
Complex tests begat complex tests. If your old test is 2-3 lines of code, I just delete it and write a new one that does the same thing.
- beny23 6y agoOh absolutely, keeping things simple should never be restricted to just production code. Testing code should be simple too. The idea that tests inhibits refactoring I find problematic though. Good tests enable refactoring. If the tests are too complex for anyone to change the code, the the code is too complex. Any complex system should be composed of simple components. Now this applies to monoliths as well as microservices. I agree it is easy to fall into the trap of thinking that tests are boring though as devs become more experienced there should be less of that thinking, hopefully.
- CraigJPerry 6y ago>> If the tests are too complex for anyone to change the code, the the code is too complex In enterprise software, the pattern i see is specifically coupling rather than garden variety complexity. There’s a really pervasive idea that unit testing means 1:1 mapping of test class to production code class. To put it another way, there’s this really common but terrible idea that tests should map to your current implementation rather than the behaviour you want. Worse, some check in test cases that should only exist locally during development, they do things like directly test private implementations. A related issue is no one ever deletes tests. Testing done well, is utterly transformational. I took a piece of enterprise software, it wasn’t badly written per se but it was hard to release. 80%+ of releases were rolled back due to issues found in production. For each release, 1 of 7 “experts” were required to review your change to this component (to make it worse it changed fairly frequently). The problem was only 2 of those 7 had a track record of only approving good changes. I’m ashamed to say i was one of the other 5 - i reviewed and approved a couple of changes that had to be rolled back from prod. So, i thought sod this and i grabbed the stingy nettles. The code was not written to be testable but fast forward 16,000 (i had a small team) test cases which execute in around 16 minutes and over the past years that software continues to be released as frequently but has NEVER had a prod rollback since. The 7 experts idea is disbanded, it’s just a normal pull request these days. Lead time on changes is about 1/3 what it was - all the faffing about exploring model output in excel for hours if not days is gone, it’s just all captured as tests. It wasn’t cheap to do this but back of the envelope calculations, it repaid within 3 months just on lack of prod outages.
- travisjungroth 6y ago> A related issue is no one ever deletes tests. I think this is a huge problem. I’m not sure why it exists. “If we have more tests it will be harder to change the code.” is an argument people make sincerely. Then... just delete the tests. If your argument against having these tests is you can imagine a point where they become a burden, just keep in mind that it’s really, really fast to delete code. “Untested” is a state we can get back to at any moment. Maybe the tests will provide a lot of value until then. Maybe if we do a big refactor we’ll delete the old tests that are too low level and make new ones. That’s fine.
- hinkley 6y agoI joke that we should make the team watch one of those Hoarder TV shows, but it's a Tears of a Clown sort of situation. Try telling people that you're deleting a flaky test and they'll stop working on their current story to try to stop you. It's as predictable as the phases of the moon.
- hinkley 6y ago> To put it another way, there’s this really common but terrible idea that tests should map to your current implementation rather than the behaviour you want. I think if you were prioritizing, then the most bang for the buck comes from making your white box tests as stupid-simple as you possibly can. Since those demand a particular implementation.
- DarkWiiPlayer 6y ago> If the tests are too complex for anyone to change the code, the the code is too complex. No, more than that, it's a sign that the tests are not just testing for how code behaves, but also its implementation details. With well-written tests, you can completely re-structure some code and, if it still does the same thing, the tests for that code will still pass. If the code being changed has several components with their own tests, then either those components have no interface change, in which case the previous paragraph applies to them, or the do, in which case their tests should be easy to adapt.
- hinkley 6y ago> Good tests enable refactoring. I don't disagree. So when I do find tests that inhibit refactoring, what do you suppose I infer about those tests?
- azeirah 6y agoI never got to understand testing. It's a thing that every developer is supposed to know and do well somehow. I always end up writing tests for my programs, but with all the stubs and fixtures and weird stuff I get into hairy problems that I don't understand. Tests are supposed to be helpful, but writing good tests is ridiculously hard. Or maybe it's just because I'm a perfectionist, and I should be happy with crappy tests that at least give me a little bit of information about regressions
- wccrawford 6y agoIt's one of those things where people who really believe in it are very vocal and will shout over anyone who dares to think there are any problems. Not everyone loves them. But, as I'm a test-lover, I have to say they're pretty amazing if you can get everyone on-board and writing good tests. As for how hard they are... It depends on how the code is structured. If the code is designed to be testable, it's pretty easy to write simple tests. Integrations get a little tougher, but as long as you know what the other side should return, it's pretty easy to stub them out. Testing existing code that was not designed to be testable is a lot harder. It typically will introduce a lot of side effects and won't be properly separated. It can be tested, but IMO just isn't worth the hassle. I've found that you end up spending as much time writing tests as you would have fixing the bugs that arise... And code doesn't get used forever. Modern dev culture insists that someone call for a rewrite eventually, no matter how good the code is, unless you're working in a system that already has decades of legacy code. My advice would be to start a small project of your own (or wait until you get a new project at work that's from scratch) and write tests from the start. It'll teach you a ton about testing and help you get the system off the ground quickly and solidly. I did this for the last big server-side system I wrote and the tests helped tremendously. The bugs that QA found were generally things that I didn't consider or that they designed wrong. The tests helped me get everything working and keep it working while they changed all kinds of things. Sadly, once that project ended up in the hands of other devs, the tests slowly died and I was the only one trying to fix them. I eventually went to another project for a while and came back to find the tests basically didn't work at all, and they died completely at that point. But they did their job at the start, and were totally worth it.