4 ms·

I am not up on all the fancy A.I. tech out there....but how different is this from something like go-fuzz which does fuzz testing based on genetic algorithms (i

I am not up on all the fancy A.I. tech out there....but how different is this from something like go-fuzz which does fuzz testing based on genetic algorithms (if my memory serves) written by Dmitry Vyukov from Google. Edit link: https://github.com/dvyukov/go-fuzz https://github.com/dvyukov/go-fuzz

jjwsteele 8y ago
Fuzz testing is different than the kind of mutation testing being referred to here. Mutation testing is about creating 'mutant' versions of your source code and determining whether your test suite detects the mutant. It is a measure similar to code coverage, in that it is a measure for determining the effectiveness of your test suite. For example, a mutation may be changing a '==' to '!='. Your test suite is then run over the mutated source code and the mutant is said to then be 'killed' if at least one test fails. This is repeated many times, each with a different mutation to your source code. Your test suite is then given a mutation score based on the number of mutants killed divided by the total mutants. Of course, there are some mutations that actually produce functionally identical code to the original. This means it isn't always possible for your test suite to kill every mutant. Because many different mutations are made, with each one resulting in your test suite being run, mutation testing can become expensive. Having only read the abstract, this looks to be about a way for determining parts of source code not worth mutating, hence reducing the amount of times your test suite needs to be run.
- Puer 8y ago
  Thank you for the clear explanation!
de_watcher 8y ago
Source code is the input data for the automated tests. Mutation testing is like "fuzzing the tests": providing a bad source code data to see if they fail to detect it.
carlsborg 8y ago
From the abstract of the referenced 1978 paper (Demillo et al): "One of the most interesting possibilities is that the mutation idea could form the basis for statistically inferring the likelihood of remaining errors in a program."
mchahn 8y ago
> ..but how different is this from ... fuzz testing Very short answer. Fuzzing changes run-time input and mutation is changing the code.

jjwsteele 8y ago

Fuzz testing is different than the kind of mutation testing being referred to here. Mutation testing is about creating 'mutant' versions of your source code and determining whether your test suite detects the mutant. It is a measure similar to code coverage, in that it is a measure for determining the effectiveness of your test suite. For example, a mutation may be changing a '==' to '!='. Your test suite is then run over the mutated source code and the mutant is said to then be 'killed' if at least one test fails. This is repeated many times, each with a different mutation to your source code. Your test suite is then given a mutation score based on the number of mutants killed divided by the total mutants. Of course, there are some mutations that actually produce functionally identical code to the original. This means it isn't always possible for your test suite to kill every mutant. Because many different mutations are made, with each one resulting in your test suite being run, mutation testing can become expensive. Having only read the abstract, this looks to be about a way for determining parts of source code not worth mutating, hence reducing the amount of times your test suite needs to be run.

Puer 8y ago
Thank you for the clear explanation!

de_watcher 8y ago

Source code is the input data for the automated tests. Mutation testing is like "fuzzing the tests": providing a bad source code data to see if they fail to detect it.

carlsborg 8y ago

From the abstract of the referenced 1978 paper (Demillo et al): "One of the most interesting possibilities is that the mutation idea could form the basis for statistically inferring the likelihood of remaining errors in a program."

mchahn 8y ago

> ..but how different is this from ... fuzz testing Very short answer. Fuzzing changes run-time input and mutation is changing the code.