3 ms·
The flakiness recommendation isn't tenable at reasonable scale. All tests are flaky to some degree. People frequently set timeouts - if your CI infra is overloa
by vlovich123 5y ago
The flakiness recommendation isn't tenable at reasonable scale. All tests are flaky to some degree. People frequently set timeouts - if your CI infra is overloaded tests, the assumptions about the timeouts will fail. Additionally, the number of non-flaky tests trends to 0 as the number of tests & engineers increases.
FB did a fantastic job here (had first hand experience before & after this system):
https://engineering.fb.com/2020/12/10/developer-tools/probabilistic-flakiness/ https://engineering.fb.com/2020/12/10/developer-tools/probab...
Wish there were products out there to be able to plug this into existing CI.