3 ms·
If you are having to refactor 150 things each time you change your codebase then maybe you need to refactor your test suite first. Direct calls in tests to cons
by jhhh 1y ago
If you are having to refactor 150 things each time you change your codebase then maybe you need to refactor your test suite first. Direct calls in tests to constructed/mocked objects is usually something you can just stuff into a private method so you only need to change it in one place.
Not quite sure I agree with the conclusion of the tiers of testing section either. If a test suite takes a long time but still covers something useful, then just deleting it because it takes too long makes no sense. Yes, if you have a 'fastTests' profile that doesn't run it that could temporarily convince you your changes are fine when they aren't. But the alternative is just never knowing your change is bad until it breaks production instead of just breaking your CI prior to that point.
- eru 1y agoA simple thing you can set up is to run short tests on every push to a feature branch, but run the long tests only when merging into master. Basically, you provisionally make the merge commit, run the expensive tests against it, and iff they pass, declare the newly created commit to be the new master.
- strogonoff 1y agoTests are code. Code has bugs. More complex code has more bugs. The more complex your tests, the more bugs in your tests. Who tests the tests? It’s one thing if you rely on functionality provided by a stable testing framework, but I bet grug no like call stacks in own test code.
- jessekv 1y ago> Who tests the tests? To me it's a bit like double entry bookkeeping. Two layers is valuable, but there's rapidly diminishing returns beyond two.
- pydry 1y agoTests get implicitly tested by being run against code. When they fail in spite of the presence of no bugs then congratulations youve found a bug in your test.
- strogonoff 1y agoWhat if a test passes as a result of the bug?
- pydry 1y agoWhat I did say: tests get tested implicitly by the code they test. What I didnt say: this catches 100% of all bugs in your tests.
- lenkite 1y agoIn my last org, we just separated "flaky" system tests into its own independent suite. They were still valuable to run - just not all the time.