3 ms·

> Code coverage has absolutely nothing to do with code quality This absolute is somewhat misplaced in my experience. Yes hitting 100% code coverage does not in

by belval 4y ago

> Code coverage has absolutely nothing to do with code quality This absolute is somewhat misplaced in my experience. Yes hitting 100% code coverage does not inherently make the code better, but it does force people to look up the tests that were there before and it often helps with providing some form of context. "Code coverage is only loosely correlated with code quality" would be my take.

retrocryptid 4y ago
Even weirder... I've come to think code coverage may be related more to security / safety than correctness. People often think they're buying safety when they go from 80% coverage to 100% coverage, and I don't think that's always true. And while it's a laudable goal, there are probably better ways to confirm safety than assuring you have 100% code coverage. We've noticed that you can still have all sorts of problems in code that has 100% coverage. We started using a TLA+ like "environment" to automagically prove assertions about our code (mostly things like "there's only one path that gets you to this counter := counter + 1 line and there's a mutex around it.") It's a pain in the rear, but probably less of a pain in the rear than trying to prove that sort of thing with traditional tests and an exhortation that you have to have 100% coverage.
BerislavLopac 4y ago
In my experience, mutation testing is an amazing tool to discover code paths not actually covered by tests, regardless of the coverage score.
cuddlybacon 4y ago
Code coverage can show the absence of good tests, but not their existence.
haspok 4y ago
Also see "Why most unit testing is waste" and "Segue" by Jim Coplien. The main takeaway is that if you test your code on the wrong ("unit") level, you may easily get ~100% coverage, but that is mostly worthless. Test on higher levels instead, ie. against business requirements, and don't worry about code coverage that much.

by belval 4y ago

retrocryptid 4y ago

Even weirder... I've come to think code coverage may be related more to security / safety than correctness. People often think they're buying safety when they go from 80% coverage to 100% coverage, and I don't think that's always true. And while it's a laudable goal, there are probably better ways to confirm safety than assuring you have 100% code coverage. We've noticed that you can still have all sorts of problems in code that has 100% coverage. We started using a TLA+ like "environment" to automagically prove assertions about our code (mostly things like "there's only one path that gets you to this counter := counter + 1 line and there's a mutex around it.") It's a pain in the rear, but probably less of a pain in the rear than trying to prove that sort of thing with traditional tests and an exhortation that you have to have 100% coverage.

BerislavLopac 4y ago

In my experience, mutation testing is an amazing tool to discover code paths not actually covered by tests, regardless of the coverage score.

cuddlybacon 4y ago

Code coverage can show the absence of good tests, but not their existence.

haspok 4y ago

Also see "Why most unit testing is waste" and "Segue" by Jim Coplien. The main takeaway is that if you test your code on the wrong ("unit") level, you may easily get ~100% coverage, but that is mostly worthless. Test on higher levels instead, ie. against business requirements, and don't worry about code coverage that much.