2 ms·

>I understand this is being used in prod, but network partitions don't occur everyday... Can't we assume they tested it before deploying to production by intro

>I understand this is being used in prod, but network partitions don't occur everyday... Can't we assume they tested it before deploying to production by introducing all the weird stuff that might happen in a distributed system like partitions, high latency, so they aren't relying only on "it seems to work in production"?

uvdn7 4y ago
My experience is that this is easier said than done. But I agree with you that in theory, one can inject failure and run tests with all possible combinations, and have a repeatable test framework (think about foundationdb’s test framework) to help root cause test failures. Sqlite has very impressive reliability; its test framework is not open sourced. It also benefits from user bug reports, as it’s literally the widest deployed db. Bugs/flaws happen. Having better tests and better observability in production are also not mutually exclusive.