3 ms·
I wasn't familiar with mutation testing. This is also from the article: >We used the open source tool PIT [35] to generate faulty versions of our programs. To
by daveguy 11y ago
I wasn't familiar with mutation testing. This is also from the article:
>We used the open source tool PIT [35] to generate faulty
versions of our programs. To describe PIT’s operation, we
must first give a brief description of mutation testing.
>A mutant is a new version of a program that is created
by making a small syntactic change to the original program.
For example, a mutant could be created by modifying a
constant, negating a branch condition, or removing a method
call. The resulting mutant may produce the same output as
the original program, in which case it is called an equivalent
mutant. For example, if the equality test in the code snippet
in Figure 1 were changed to if (index >= 10), the new
program would be an equivalent mutant.
Sounds like a resource intensive way to test -- # tests * loc * mutation options. However, if you need the fault tolerance it could be worth it. You can probably spot check to get similar quality with lower resources. Very interesting.
- julian37 11y agopitest has a few tricks up its sleeve to keep overhead reasonable. For example (from http://pitest.org/faq/ http://pitest.org/faq/): > Per test case line coverage information is first gathered and all tests that do not exercise the mutated line of code are disguarded. I've used pitest in real life and while it is quite a bit heavier than straightforward testing, it's definitely usable. I'm now using it on all Java projects, I wish there was something comparable for JavaScript!
- mordocai 11y agoI don't think it is quite as complete, but this looks usable https://github.com/jimivdw/grunt-mutation-testing https://github.com/jimivdw/grunt-mutation-testing. That's just from google, no experience with it.
- lmmi 11y agoComputing the mutation score using a randomly selected subset of the mutants seems to work well, though it hasn't been studied a lot. One paper is "Reducing the cost of mutation testing: An empirical study" by Wong and Mathur.