4 ms·

There are plenty of places where inverting a condition won't matter. Anything that just requires a consistent ordering, for one.

the_af 11y ago
Your tests should be written in a way that they test there is a consistent ordering, if that's a relevant property for you. If they don't, you have a problem. If they do, I guess the mutation framework will merely point out your tests don't detect some changes; it doesn't necessarily mean they are bad tests, it's merely a "smell" that deserves further investigation. (By the way, I think you're overstating your case when you claim "there are plenty of places" where it doesn't matter. I think in the vast majority of cases, if you change a "<" to a ">", you introduce a bug!)
- TheLoneWolfling 11y ago
  I agree that in the majority of cases it will cause bugs. But what I'm saying is that there are cases where it can legitimately not cause a bug. The problem I have is this: this testing framework explicitly states the following: > The quality of your tests can be gauged from the percentage of mutations killed So, you write your tests to test that there is a consistent ordering, it changes the "<" to a ">", finds that all tests still pass (because, as you said, you're only testing if there is a consistent ordering, because that's the only thing that matters in this case), and considers your tests lower-quality because of that. This is a flaw most mutation testing - and, for that matter, test driven design - suffers. For another flaw: there are many things that end up along the lines of a bunch of shortcut "fast paths" at the start, with a slow path at the end that does slow but exhaustive checking. For example, if you're checking that two objects are equal you may insert a check at the start to see if they are the same pointer, and if so return `true`. If you're doing division in software you may check if you're dividing by 1 instead of doing the computation. That sort of thing. Any mutation that introduces false negatives into the shortcuts (but no false positives) won't be flagged by mutation testing.

the_af 11y ago

Your tests should be written in a way that they test there is a consistent ordering, if that's a relevant property for you. If they don't, you have a problem. If they do, I guess the mutation framework will merely point out your tests don't detect some changes; it doesn't necessarily mean they are bad tests, it's merely a "smell" that deserves further investigation. (By the way, I think you're overstating your case when you claim "there are plenty of places" where it doesn't matter. I think in the vast majority of cases, if you change a "<" to a ">", you introduce a bug!)

TheLoneWolfling 11y ago
I agree that in the majority of cases it will cause bugs. But what I'm saying is that there are cases where it can legitimately not cause a bug. The problem I have is this: this testing framework explicitly states the following: > The quality of your tests can be gauged from the percentage of mutations killed So, you write your tests to test that there is a consistent ordering, it changes the "<" to a ">", finds that all tests still pass (because, as you said, you're only testing if there is a consistent ordering, because that's the only thing that matters in this case), and considers your tests lower-quality because of that. This is a flaw most mutation testing - and, for that matter, test driven design - suffers. For another flaw: there are many things that end up along the lines of a bunch of shortcut "fast paths" at the start, with a slow path at the end that does slow but exhaustive checking. For example, if you're checking that two objects are equal you may insert a check at the start to see if they are the same pointer, and if so return `true`. If you're doing division in software you may check if you're dividing by 1 instead of doing the computation. That sort of thing. Any mutation that introduces false negatives into the shortcuts (but no false positives) won't be flagged by mutation testing.