4 ms·
I'd definitely say that the absence of coverage is a problem. My view is that coverage is necessary but not sufficient for good testing. We talked a bit about
by lmmi 11y ago
I'd definitely say that the absence of coverage is a problem. My view is that coverage is necessary but not sufficient for good testing.
We talked a bit about classifying tests but didn't do it in the end because it's surprisingly hard to do. I do know of one paper that looked at different kinds of tests, called "The Effect of Code Coverage on Fault Detection under
Different Testing Profiles": http://goo.gl/nnxgwE http://goo.gl/nnxgwE. The authors found differences between tests for error cases vs. tests for normal operation and between functional tests vs. random tests. IIRC, they had undergrads do a term project that had to pass 1200 tests before the final submission, and the professors themselves wrote the tests, so categorization was a bit easier.
- jacques_chester 11y agoAs a rough way to automatically classify tests, you can look for tools like capybara, selenium or htmlunit for feature and mocking libraries for unit. Mind you, there's as many taxonomies for tests as there are tests. To be honest I expect one way to classify them is by working backwards from coverage -- feature tests should have low volume but wide distribution across a codebase (perhaps that's another metric -- density?). Unit tests would be narrow but deep on a particular module.
- lmmi 11y agoThose are good ideas, thanks!
- jacques_chester 11y agoThrow me on as the tenth or eleventh coauthor and I'll buy beer to sweeten the deal. If you need a corpus of code from a highly doctrinaire TDD shop, or if you think we can help, let me know: jchester@pivotal.io.