3 ms·
I was going to be worried if mutation testing wasn't mentioned here. Is a great way to test the effectiveness of your tests at catching the common mistakes peo
by taeric 5y ago
I was going to be worried if mutation testing wasn't mentioned here. Is a great way to test the effectiveness of your tests at catching the common mistakes people make in code.
That is, a mutation suit doesn't test your code, per se. It tests your test suit.
- sverhagen 5y agoI've worked a lot on Java, and while there is great mutation testing tools (well, PIT specifically), I find them hard to "scale" practically, i.e. to run them in an automated fashion "every time". You either get long running builds with them in it, or you have to deal with the logistics of moving caches around (so that the mutation testing can be incremental). And as just another tool that a developer MAY use if they so choose to, it's... well... nice... but no longer front of mind, for most people on the team. I'm very interested if you're experiences have been different?
- taeric 5y agoNo, I found the same. Team disabled them on every release build, as they felt it was too slow. I may bring them back in the release servers, but it is honestly trying to argue for these when nobody else cares.
- masklinn 5y agoIt needs someone who cares which us a thankless job but it’s useful as smote tests run nightly / weekly. It tends to be better for broad concerns which trigger somewhat rarely, tend not to be otherwise caught, but can be quite concerning. Crater is a good example of that, it’s infeasible to build & test every public crate on every CI check, but it’s a useful sanity check when release arrives or when messing with the more subtle aspects of the compiler.