4 ms·
It helps to ask "what is most likely to change about the code I am testing?" when writing unit tests. Try to avoid testing those things, and instead focus on te
by nevir 8y ago
It helps to ask "what is most likely to change about the code I am testing?" when writing unit tests. Try to avoid testing those things, and instead focus on testing the code's behavior (quack)
- hinkley 8y agoAnd then some pinhead starts cheerleading 100% code coverage. Usually long before you’ve gotten your test quality high enough to be sustainable.
- adrianratnapala 8y agoWhat I have trouble with is even gauging the quality of existing tests.
- crdoconnor 8y agoIf you track failures and categorizing them as either failures that caught a bug or false positives then you basically have a good measure for test quality.
- hinkley 8y agoGood tests are hard. Beware of anyone who tries to pretend otherwise. To build on what crdoconnor said, empirical evidence helps. Do a post mortem every time. Try to recall what was going on when the problem started and see if there are any reasonable measures you can undertake in the future. It's the typical iterative process of learning something as you go along. When you discover you made a mistake, try to figure out how not to make it again. Every time you get a regression in the code ask yourself "didn't we have a test for that?" If the answer is 'no' then think back to the last time someone (maybe you!) talked you out of spending more time on tests. Is the hassle now worth the hassle you avoided then? Occasionally this is the case, and the response is "shit happens" (if this is not rare, then you might need an attitude adjustment). If you find that prevention would have been better, try to remember that next time you're thinking about cutting back on tests. If the answer was 'yes', then something has gone wrong with your testing strategy. Was it always broken, or did subsequent code changes break it? When you fix the test(s), would the way they look now have made sense at the time you wrote them to begin with? Should new tests you write look more like these do? Should you go back right now and fix other tests to make them look this way? Just keep thinking. Verify your assumptions. Chuck the ones that don't hold water. Watch how other people deal with similar problems, and steal their ideas (or learn from their mistakes).
- adrianN 8y agoMutation testing is a good way to assess test quality. You ideally want every mutation to be caught be a very small number of tests.
- aiCeivi9 8y agoAs far I can tell pi-test can't scale even to medium sized projects, time complexity is close to (unit tests * classes). Are the other tools (maybe for other languages) that deal with it better, without the need to split code in smaller parts?
- _pmf_ 8y ago> And then some pinhead starts cheerleading 100% code coverage. I've seem IBM consultants increasing test coverage from 40% to 90% without creating any new test assertions (in fact deleting existing test assertions) because the contract stated test coverage as only goal.
- hinkley 8y agoIt may not be the worst metric dysfunction we encounter at work, but it's the one we have the most control over, so it feels like a self-inflicted wound even when it's your coworkers doing it.