4 ms·
If every test assumes the same invariant, then no test ever covers cases where the invariant doesn't hold. I've seen mature teams get this wrong even for basic
by fl0ki 2y ago
If every test assumes the same invariant, then no test ever covers cases where the invariant doesn't hold.
I've seen mature teams get this wrong even for basic things. For example, they had a factor option, but all of their tests only used a factor of 1.0 -- applying the factor 0 or 2+ times didn't fail any tests, the factor was effectively completely untested despite "coverage".
My tests caught it because I was a dependent of their code and, frankly, trusted them much less than they trusted themselves. Rightly in this case, and it turned out, many others.
My philosophy was to test with exactly the configuration to be used in production and as much as possible [representative subsets of] the data to be used in production. Their philosophy was only to test contrived isolated cases which proved grossly inadequate with respect to production.