4 ms·
We once accidentally made a change to a python project test suite that caused it to successfully run none of the tests. Then we broke some stuff but the tests k
by singron 2y ago
We once accidentally made a change to a python project test suite that caused it to successfully run none of the tests. Then we broke some stuff but the tests kept "passing".
It's a little difficult to productionize an always_fail test since you do actually want the test suite to succeed. You could affirmatively test that you have non-zero passing tests, which is I think what we did. If you have an always_fail test, you could check that that's your only failure, but you have to be super careful that your test suite doesn't stop after a failure.
- maxbond 2y agoMaybe you could ignore that test by default, and then write a shell script to run your tests in two stages. First you run only the should-fail test(s) and assert that they fail. Then you can run your actual tests.
- SoftTalker 2y agoSounds like the old George Carlin one-liner. Or maybe it's a two-liner: The following statement is true. The preceeding statement is false.
- robotresearcher 2y agoEven older than George Carlin. The Liar Paradox is documented from at least 400BC. https://en.m.wikipedia.org/wiki/Liar_paradox https://en.m.wikipedia.org/wiki/Liar_paradox
- maxbond 2y agoI have to imagine it's about as old as propositional logic (so, as old as the hills). I most closely associate it with Gödel and his work on incompleteness.
- marcosdumay 2y ago> We once accidentally made a change to a python project test suite that caused it to successfully run none of the tests. That shouldn't be an easy mistake to make. Your test code should be clearly marked, and better if slightly separated from the rest of the code. Also, there should be some feedback about the amount of tests that run. And yeah, I know Python doesn't help you make those things.