5 ms·

"Code coverage measures how comprehensively a piece of code has been tested; if a program contains 100 lines of code and the quality-assurance process tests 95

"Code coverage measures how comprehensively a piece of code has been tested; if a program contains 100 lines of code and the quality-assurance process tests 95 lines, the effective code coverage is 95 percent." I've never heard of this before. Where does this come from and why did they suspect this would be indicative of anything?

aout 12y ago
It is a fairly common practice in companies that do not have technical managers (http://en.wikipedia.org/wiki/Code_coverage http://en.wikipedia.org/wiki/Code_coverage). Basically it is useful to measure the coverage of existing conditions (and insure stability) but clearly lacks the ability to test the conditions you forgot to write. Kind of defeats half of the purpose.
- robfblair 12y ago
  Companies also track and report is when required by external entities such as the FAA for safety critical systems. I think it goes deeper than "the conditions you forgot to write" although that's part of it. A developer's test suite tends to make assumptions and these are the same assumptions that exist in the production code. If your assumptions about usage, or input, or edge cases are poor then the tests you write will be poor too and in that case 100% coverage isn't worth much. Big reason I'm a fan of things like QuickCheck in Haskell.
  - zelos 12y ago
    A quick Google suggests the FAA uses some version of decision coverage (i.e. for each branch in your code, what percentage were taken during testing), not simple line coverage.
cgh 12y ago
The idea is that shipping untested code is worse than shipping (unit) tested code. Code coverage reporting is quite common and I'm surprised you've never even heard of it if you've ever written software professionally.
- tbrownaw 12y ago
  It's popular among the hobbyist / activist crowd. People who like to talk about the "proper" way to develop software tend to like it. People who run open source projects tend to either like it or bow to peer pressure. But people working on in-house and enterprisey stuff? Yeah, not so much there.
  - KMag 12y ago
    > But people working on in-house and enterprisey stuff? Yeah, not so much there. This depends a lot on your corporate development culture, even for in-house enterprisey software.
  - jacquesm 12y ago
    > But people working on in-house and enterprisey stuff? Yeah, not so much there. I suspect you are working from a limited sample size there.
  - cgh 12y ago
    "But people working on in-house and enterprisey stuff? Yeah, not so much there." That is absolutely incorrect, in my experience.
dankohn1 12y ago
Take a look at tools like Istanbul for Node.js and Simplecov for Ruby on Rails. https://coveralls.io/ https://coveralls.io/ has a nice list of code coverage tools for many languages. If you don't have tests, it is an extremely manual process to change anything about your codebase (including dependencies) and have any confidence that you haven't broken things. Once you write tests, code coverage lets you figure out what code is not being tested.
wyldfire 12y ago
> Where does this come from It's defining the term "code coverage" to be the ratio of lines covered by tests to the total lines. > why did they suspect this would be indicative of anything? If anything, it's a useful feedback mechanism. e.g. "Gee, we wrote sixteen unit tests for this module but we're still only at 34% coverage. Ohh, I see it now -- framistan.cpp doesn't get used by these tests at all. We should write a test that enables framistan mode!"
jacquesm 12y ago
Checking your coverage for tests is a pretty old concept. Early 1960's or so!

aout 12y ago

It is a fairly common practice in companies that do not have technical managers (http://en.wikipedia.org/wiki/Code_coverage http://en.wikipedia.org/wiki/Code_coverage). Basically it is useful to measure the coverage of existing conditions (and insure stability) but clearly lacks the ability to test the conditions you forgot to write. Kind of defeats half of the purpose.

robfblair 12y ago
Companies also track and report is when required by external entities such as the FAA for safety critical systems. I think it goes deeper than "the conditions you forgot to write" although that's part of it. A developer's test suite tends to make assumptions and these are the same assumptions that exist in the production code. If your assumptions about usage, or input, or edge cases are poor then the tests you write will be poor too and in that case 100% coverage isn't worth much. Big reason I'm a fan of things like QuickCheck in Haskell.
- zelos 12y ago
  A quick Google suggests the FAA uses some version of decision coverage (i.e. for each branch in your code, what percentage were taken during testing), not simple line coverage.

cgh 12y ago

The idea is that shipping untested code is worse than shipping (unit) tested code. Code coverage reporting is quite common and I'm surprised you've never even heard of it if you've ever written software professionally.

tbrownaw 12y ago
It's popular among the hobbyist / activist crowd. People who like to talk about the "proper" way to develop software tend to like it. People who run open source projects tend to either like it or bow to peer pressure. But people working on in-house and enterprisey stuff? Yeah, not so much there.
- KMag 12y ago
  > But people working on in-house and enterprisey stuff? Yeah, not so much there. This depends a lot on your corporate development culture, even for in-house enterprisey software.
- jacquesm 12y ago
  > But people working on in-house and enterprisey stuff? Yeah, not so much there. I suspect you are working from a limited sample size there.
- cgh 12y ago
  "But people working on in-house and enterprisey stuff? Yeah, not so much there." That is absolutely incorrect, in my experience.

dankohn1 12y ago

Take a look at tools like Istanbul for Node.js and Simplecov for Ruby on Rails. https://coveralls.io/ https://coveralls.io/ has a nice list of code coverage tools for many languages. If you don't have tests, it is an extremely manual process to change anything about your codebase (including dependencies) and have any confidence that you haven't broken things. Once you write tests, code coverage lets you figure out what code is not being tested.

wyldfire 12y ago

> Where does this come from It's defining the term "code coverage" to be the ratio of lines covered by tests to the total lines. > why did they suspect this would be indicative of anything? If anything, it's a useful feedback mechanism. e.g. "Gee, we wrote sixteen unit tests for this module but we're still only at 34% coverage. Ohh, I see it now -- framistan.cpp doesn't get used by these tests at all. We should write a test that enables framistan mode!"

jacquesm 12y ago

Checking your coverage for tests is a pretty old concept. Early 1960's or so!