3 ms·
Flakiness isn’t always because of the code under test. CI systems can have any number of flakiness issues way before they ever get a chance to run your code. S
by ninkendo 4y ago
Flakiness isn’t always because of the code under test. CI systems can have any number of flakiness issues way before they ever get a chance to run your code. Spinning up VM’s/containers, installing build dependencies, SDK’s, huge amount of prerequisites to even running your code, clusters of test runners that can fall over, bad connectivity, etc etc etc.
Take a look at the kubernetes/kubernetes repo and look for kind/flake to get an idea of what a project looks like that has a tremendously complex integration testing, and how often the integration tests aren’t 100% reliable. I’d wager any organization that’s “all-up”/“end-to-end” testing software past a certain level of complexity (especially anything that touches cloud service integration) is going to suffer this problem. If you’re a developer on such a project, flakes are just sorta the way of things.