> Another more out-there example is the disabling of tests over time as test coverage starts to overlap. A lot of tests in a large codebase exercise the same co

> Another more out-there example is the disabling of tests over time as test coverage starts to overlap. A lot of tests in a large codebase exercise the same codepaths, and this is usually not a net benefit (it sometimes can be, eg if one test runs really fast vs another more comprehensive one thats slow). So having a way to evaluate whether a test is actionable or useful is something that emerges organically as large codebases begin to change. If the cost of maintaining these tests exceeds the value they provide, don’t just disable them: delete them. But if the maintenance isn’t an issue, and it’s just the compute resources that are an issue, keep them enabled and schedule your tests more effectively! For example, at my company we have tests which are run the moment you want to commit, and will block your commit if they fail, and then we have a separate service which continually runs tests against tip 24/7. In order to keep low friction in the dev process, we only run a subset of tests as part of the commit process, and assume that a few failures will make it into tip but will be caught within a couple days (we ship every two weeks so that timing’s been pretty safe).