3 ms·

Whenever I see a test suite do 9 steps of setup to assert one thing and then (mostly) the same 9 steps again to assert some other thing, I die a little inside.

Whenever I see a test suite do 9 steps of setup to assert one thing and then (mostly) the same 9 steps again to assert some other thing, I die a little inside. Especially when the setup takes multiple seconds for each case. The lesser evil is to just ”do what you need and test everything once you are arranged”. You won’t get hundreds of neatly separated well-named test cases which fail for a single reason. But for slow tests that isn’t as important as keeping the redundant setup away. I like the tree idea but once we have simple pure/immutable we don’t really have the problem of redundant setup being slow, just ugly.

sevenK8s 2y ago
How do you “test once arranged” as your test are modifying things that conflict? Setup: login. Test 1: delete your account. Test 2: can change username. The last time I saw this handled was a test copy of db per thread and transactional test rolled back. Not great but it did 10x our pipelines and avoided locks and issues.
- alkonaut 2y ago
  If they conflict then it's a separate "story". They couldn't be part of the same timeline. You don't end up with one big ball of mud test which tests everything in the app, you end up with N different tests based on what setup they need, each one doing as many asserts as is supported by that setup. Here it might be test1: delete account. Test2: test changing things on the account (email, profile picture, ...) and asserting that each of the changes work. BUT obviously you can just test deleting the account at the end of the test that modifies the account. SetAccountUserName(account, "New name"); GetAccount(account.id).UserName.Should().Be("New name"); SetAccountEmail(account, "new@email"); GetAccount(account.id).Email.Should().Be("new@email"); DeleteAccount(account); GetAccount(account.id).Should().BeNull(); This is a working timeline
int_19h 2y ago
Note that the tree concept is orthogonal between writing and running - tests written like a tree can just as well desugar into a linear list of all possible combinations, re-running the setup steps every time.
__MatrixMan__ 2y ago
A related evil is when people start with a test that orchestrates the whole flow, copy it, and make one small change towards the beginning of the flow. They know, at the time of doing this, that the part they care about happens after one second. They could just let it crash and burn after making the critical assertion and the test would be just as useful. But they commit an entirely new test filled with redundant stuff that takes way longer than is necessary and makes it unclear which assertion was the critical one. Because hey, look at all that green text in my PR. I'm so thorough.

sevenK8s 2y ago

How do you “test once arranged” as your test are modifying things that conflict? Setup: login. Test 1: delete your account. Test 2: can change username. The last time I saw this handled was a test copy of db per thread and transactional test rolled back. Not great but it did 10x our pipelines and avoided locks and issues.

alkonaut 2y ago
If they conflict then it's a separate "story". They couldn't be part of the same timeline. You don't end up with one big ball of mud test which tests everything in the app, you end up with N different tests based on what setup they need, each one doing as many asserts as is supported by that setup. Here it might be test1: delete account. Test2: test changing things on the account (email, profile picture, ...) and asserting that each of the changes work. BUT obviously you can just test deleting the account at the end of the test that modifies the account. SetAccountUserName(account, "New name"); GetAccount(account.id).UserName.Should().Be("New name"); SetAccountEmail(account, "new@email"); GetAccount(account.id).Email.Should().Be("new@email"); DeleteAccount(account); GetAccount(account.id).Should().BeNull(); This is a working timeline

int_19h 2y ago

Note that the tree concept is orthogonal between writing and running - tests written like a tree can just as well desugar into a linear list of all possible combinations, re-running the setup steps every time.

__MatrixMan__ 2y ago

A related evil is when people start with a test that orchestrates the whole flow, copy it, and make one small change towards the beginning of the flow. They know, at the time of doing this, that the part they care about happens after one second. They could just let it crash and burn after making the critical assertion and the test would be just as useful. But they commit an entirely new test filled with redundant stuff that takes way longer than is necessary and makes it unclear which assertion was the critical one. Because hey, look at all that green text in my PR. I'm so thorough.