3 ms·
Interesting thanks for the link! I’m curious how many tests were in the small size range for that chart which provides evidence to show the size-flakiness corr
by breatheoften 9y ago
Interesting thanks for the link!
I’m curious how many tests were in the small size range for that chart which provides evidence to show the size-flakiness correlation holds in tests that use tools associated with higher than average flakiness...
I’m also feeling like I want to have more clarity around the mechanism for measuring flakiness — the definition they use is that a test is flakey if it shows both failing and success runs with the “same code” — does “same code” refer to a freeze of only the codebase under test or also a statement about change to the tools in the testing environment ...?
I wonder what the test suites for tools like selenium/WebDriver look like ... do they track a concept of “meta-flakiness” to try and observe changes to test flakiness results caused by changes to the test tooling ...?
- Vinnl 9y agoYeah, good questions, the post leaves some to be desired. And meta-flakiness tooling actually sounds like it could be really useful!