3 ms·
I'm not an expert on this topic, but I don't think those are the kinds of changes generated by mutation testing (or at least, not the only ones). A more interes
by the_af 12y ago
I'm not an expert on this topic, but I don't think those are the kinds of changes generated by mutation testing (or at least, not the only ones). A more interesting example of a mutation is changing a "<" to a ">" in a condition. If your tests don't detect this, you're in trouble...
- TheLoneWolfling 12y agoThere are plenty of places where inverting a condition won't matter. Anything that just requires a consistent ordering, for one.
- the_af 12y agoYour tests should be written in a way that they test there is a consistent ordering, if that's a relevant property for you. If they don't, you have a problem. If they do, I guess the mutation framework will merely point out your tests don't detect some changes; it doesn't necessarily mean they are bad tests, it's merely a "smell" that deserves further investigation. (By the way, I think you're overstating your case when you claim "there are plenty of places" where it doesn't matter. I think in the vast majority of cases, if you change a "<" to a ">", you introduce a bug!)
- TheLoneWolfling 12y agoI agree that in the majority of cases it will cause bugs. But what I'm saying is that there are cases where it can legitimately not cause a bug. The problem I have is this: this testing framework explicitly states the following: > The quality of your tests can be gauged from the percentage of mutations killed So, you write your tests to test that there is a consistent ordering, it changes the "<" to a ">", finds that all tests still pass (because, as you said, you're only testing if there is a consistent ordering, because that's the only thing that matters in this case), and considers your tests lower-quality because of that. This is a flaw most mutation testing - and, for that matter, test driven design - suffers. For another flaw: there are many things that end up along the lines of a bunch of shortcut "fast paths" at the start, with a slow path at the end that does slow but exhaustive checking. For example, if you're checking that two objects are equal you may insert a check at the start to see if they are the same pointer, and if so return `true`. If you're doing division in software you may check if you're dividing by 1 instead of doing the computation. That sort of thing. Any mutation that introduces false negatives into the shortcuts (but no false positives) won't be flagged by mutation testing.