2 ms·
Stable and meaningful tests are very hard to write. Probably harder than writing the software that gets tested. It’s even harder to maintain them and keep meani
by Ididntdothis 6y ago
Stable and meaningful tests are very hard to write. Probably harder than writing the software that gets tested. It’s even harder to maintain them and keep meaningful. Tests that made sense a while ago may become flaky or obsolete when the system changes.
- 2rsf 6y agoStatistics is simply working against you, system tests are a long chain of of (let's say) independent events with (let's hope) a small chance for failure. But your test multiply those probabilities creating a much bigger probability for failure. Now you run multiple tests, each with a more complex environment than your real system (you need to control your tests, test logs etc.) usually on a lesser environment than your production one. Add all this and you'll get an uncomfortable probability of failure
- abjKT26nO8 6y ago> But your test multiply those probabilities creating a much bigger probability for failure. Minor nitpick: when probabilities multiply, they get smaller. Such is the nature of numbers in the range [0, 1]. Multiplication would happen when counting the probability of two (independent) events coinciding. What you're thinking about here is the probability of any one of several events occuring. That will be a (rather convoluted) sum, not a product.
- 2rsf 6y agoWe engineers can do convoluted sums, the assumption is that the events are independent and you multiply the probability of success (or 1-probability of failure).