I agree that this is a real and separate problem, but I believe the solution lies outside of the test suite. One way I have seen this handled is to enforce res

I agree that this is a real and separate problem, but I believe the solution lies outside of the test suite. One way I have seen this handled is to enforce restricting rollouts of a feature flag to 95% at most. That way turning a feature all the way on requires removing the flag from your codebase. It’s draconian, but honestly anything less than that leads to the situation you describe.