That's a great point! Depends on the what you define by "externality". I like to look at it this way, what's being tested? I see three groups: - testing th

That's a great point! Depends on the what you define by "externality". I like to look at it this way, what's being tested? I see three groups: - testing the software you wrote. - testing integration with other software you wrote (microservice interaction maybe?) - testing integration to third party software. I'm telling you our current plan, feel free to poke holes: In the first case the only externalities that seem to really matter are mostly resource and configuration files. We could detect I/O with such files and rerun the test if the underlying resource/config changed. In the second case were planning on collecting coverage for all services being tested and then employ a more sophisticated diff heurastic. For the third case mark those tests as always run, hopefully you are testing your codebase and not just thirdparty integrations. We have a poc of a syscall tracer that gives us an I/O map of the test being run or the software being tested. this includes reading/writing to files, pipes, network etc...