4 ms·
Automated code reviews via mutation testing
- swader999 1y agoFascinating idea. The price stopped me right in my tracks.
- breppp 1y agohttps://testing.googleblog.com/2021/04/mutation-testing.html https://testing.googleblog.com/2021/04/mutation-testing.html https://research.google/pubs/state-of-mutation-testing-at-google/ https://research.google/pubs/state-of-mutation-testing-at-go...
- mkw5053 1y agoMy arc of using AI code generation tools started by steadily adding more and more guardrails. I realized the returns were diminishing while cost, complexity, and latency climbed. Mutation testing made it obvious I had gone too far. While there is a time and place for it, I completely lost the flow and was fighting against my dev setup. I’ve since found a better balance: the most strict, opinionated dev environment, especially on types and linting, with the least code and configuration.
- looperhacks 1y agoYears ago, I used mutation testing in a pretty big Java project. Let me tell you: You don't want so much coverage. It's usually a waste of time and makes changes to existing code very annoying. Keep mutation testing/high coverage for a small selection of _very important_ code, but don't overdo it
- pfdietz 1y agoBut did it lead to the finding of additional bugs?
- looperhacks 1y agoIt's very likely that over the years, it found something. But we didn't have anything to compare against, because it was always mutation testing from day one.
- Xss3 1y agoWhy are you making Europeans pay ~15% more for the same product? Instant nope from me on that basis alone.
- Xss3 1y agoYou open this thread and dont reply to any questions? Where is the respect in that? Gross
- aDyslecticCrow 1y agoI read about mutation testing in a course of test oriented programming. And i am unsure to this day of what domain it's meant for. The most strict and safety critical code i have encountered, was specifying its system behaviour in a state machine diagram of a ISO specification. Yet it was too state based for effective unit testing. Instead it opted to assert the living hell out of the codebase and be paranoid about even the stability of the ram it wrote to. I have a hard time imagining a domain that is both; 1. Easily to coverage test to a high enough degree that mutating testing can be used. 2. Needs pedantic testing to the point of absurdity, perhaps already using boundary value diagrams for its test-cases. And so mutating testing feels quite odd to me, and selling it as an AI tooling is even more insane. It feels like far more effort than to just not use AI.
- pfdietz 1y agoPaper from 2017 on mutation testing for code review at Google: https://research.google/pubs/state-of-mutation-testing-at-google/ https://research.google/pubs/state-of-mutation-testing-at-go...