3 ms·

Sometimes, it takes a lot less code to test a particular code by doing it in a long test with multiple assertions. Migrate up -> assert ok -> rollback 1 -> ass

Sometimes, it takes a lot less code to test a particular code by doing it in a long test with multiple assertions. Migrate up -> assert ok -> rollback 1 -> assert ok -> rollback 2 -> assert ok I don’t see much benefit to breaking it up, and you’re testing state changes between each transition, so the entire test is useful and simpler, shorter, and clearer than the alternative.

twic 4y ago
Sometimes it's also more meaningful to the reader this way. Imagine an object which sees numbers, tries to pair up identical numbers, and reports the set of unpaired numbers. A good test would be: Create object Assert it has no unpaired numbers Show it 1, 2, and 3 Assert it has 1, 2, and 3 as the unpaired numbers Show it 1 and 3 Assert it has 2 as the unpaired number This test directly illustrates how the state changes over time. You could split it into three tests, but then someone reading the tests would have to read all three and infer what is going on. I consider that strictly worse.
TrianguloY 4y ago
That test is asserting a flow, not features. How can you be sure that the second rollback fails because that rollback code is wrong and not because the previous two functions made some unexpected changes? Or rather, how can you be sure that the second rollback is ok if you dont know the state it was run from? Maybe the migration set an unexpected flag that made the rollbacks pass that test without working properly. This is also the reason why tests should be run in arbitrary order, to avoid unexpected interactions due to order. Flow tests can be useful in some situations, but they should never replace individual feature tests.
seadan83 4y ago
This can be a path where things do go bad. Let's say thus test pattern is a success and then is replicated for many tests. Now, the schema or migration changes. A small change there now breaks the entire test suite. At this point the number of failing tests only indicates how many hours you will be fixing assertions. Another failure mode is when test scaffolding builds up. Imagine that migrate up part becoming multiple schemas, or services. It then fails, now finding exactly where to fix the test scaffolding becomes a multi-hit exercise. I'm not saying the example is bad, but it can put you on a path where if you constantly build on top of it, it can bad (eg, developers that don't care for tests nor test code quality, or just want to go home, and they just add a few assertions, add some scaffolding, copy-paste it all and mutate some assertions for a different table & rinse-wash-repeat across 4 people, 40 hours a week for 3 years...)
- christophilus 4y ago
  Sorry the example was vague. It was a test for the migration library itself— something I wrote recently when playing around with building a 0-dependency web framework for Bun. I wouldn’t actually write tests for migrations themselves.

twic 4y ago

Sometimes it's also more meaningful to the reader this way. Imagine an object which sees numbers, tries to pair up identical numbers, and reports the set of unpaired numbers. A good test would be: Create object Assert it has no unpaired numbers Show it 1, 2, and 3 Assert it has 1, 2, and 3 as the unpaired numbers Show it 1 and 3 Assert it has 2 as the unpaired number This test directly illustrates how the state changes over time. You could split it into three tests, but then someone reading the tests would have to read all three and infer what is going on. I consider that strictly worse.

TrianguloY 4y ago

That test is asserting a flow, not features. How can you be sure that the second rollback fails because that rollback code is wrong and not because the previous two functions made some unexpected changes? Or rather, how can you be sure that the second rollback is ok if you dont know the state it was run from? Maybe the migration set an unexpected flag that made the rollbacks pass that test without working properly. This is also the reason why tests should be run in arbitrary order, to avoid unexpected interactions due to order. Flow tests can be useful in some situations, but they should never replace individual feature tests.

seadan83 4y ago

This can be a path where things do go bad. Let's say thus test pattern is a success and then is replicated for many tests. Now, the schema or migration changes. A small change there now breaks the entire test suite. At this point the number of failing tests only indicates how many hours you will be fixing assertions. Another failure mode is when test scaffolding builds up. Imagine that migrate up part becoming multiple schemas, or services. It then fails, now finding exactly where to fix the test scaffolding becomes a multi-hit exercise. I'm not saying the example is bad, but it can put you on a path where if you constantly build on top of it, it can bad (eg, developers that don't care for tests nor test code quality, or just want to go home, and they just add a few assertions, add some scaffolding, copy-paste it all and mutate some assertions for a different table & rinse-wash-repeat across 4 people, 40 hours a week for 3 years...)

christophilus 4y ago
Sorry the example was vague. It was a test for the migration library itself— something I wrote recently when playing around with building a 0-dependency web framework for Bun. I wouldn’t actually write tests for migrations themselves.