4 ms·

> Deleting test1 loses us another property from the Test Desiderata—tests should be specific. That’s the property of tests where, when one fails, you know exact

> Deleting test1 loses us another property from the Test Desiderata—tests should be specific. That’s the property of tests where, when one fails, you know exactly where the problem is. Contra Kent and, it seems, prevailing wisdom, I think simply deleting test1 is by far the simplest, clearest and best way. Provided that your testing framework tells you which specific assertion failed (e.g., by telling you the line number in a stack trace), you do know exactly where the problem is. The only thing you lose is that a test function or method may now cover several related checks (they are related by "setup dependence"), meaning their names may need to be somewhat broader. But you can still describe the specific semantics of each assert() check in a one-line comment beforehand if you want. There's no need to cram it into a legal method name. ETA: Prefer to write tests whose "arrange" steps are as simple as possible, to minimise unnecessary overlaps. But if the simplest possible "arrange" step for a test is something that itself needs to be checked for correctness, just do that check right there, and nowhere else. Anything beyond that is ceremony that adds nothing useful.

skydhash 2mo ago
> Contra Kent and, it seems, prevailing wisdom, I think simply deleting test1 is by far the simplest, clearest and best way. Provided that your testing framework tells you which specific assertion failed (e.g., by telling you the line number in a stack trace), you do know exactly where the problem is. The issue with your approach is that codepath execution is more of a graph than a linear timeline. With one test, you artificially constrains it to a linear timeline even if the assertions are correct. With multiples and independent you only assert one specific node. That lets you switch up how you do the preliminary steps. I much prefer an exhaustive test unit for step 1, and just a regular call in step 2 and step 3.
- akoboldfrying 2mo ago
  If you need test1's logic as setup for test2, then you already have that "linear timeline" -- for test2 all by itself. Additionally (that is, redundantly) running the same setup code "prefix" by itself in test1 doesn't remove test2's linear timeline. ETA: I'm assuming your objection to a "linear timeline" is that it reduces the potential for running tests in parallel -- have I got that right? If not, what do you see as being the problem with it?
  - skydhash 2mo ago
    If I know that step 1 is correct for all the combinations (N) of its input (in test 1), in test 2, I only need to test the specific combinations (M) of step 2, treating step 1 as an axiom. So no need to have NxM in one test, or NxM tests. Like if you were testing a drone stabilization software, testing the flying state can always assume that it has indeed taken off. No need to validate that it has done so for each scenarios, I can directly put it the correct value. It’s a contrived example, but that axiomatic aspect helps greatly when designing interfaces to reduce coupling between step 1 and step 2.
dack 2mo ago
I think this could lead to the tests getting harder to reason about over time because individual test size will just grow (many iterations of deleting the simple test in favor of a more complicated test that also asserts the simple things) and if you start simplifying the more complicated test later, you may not realize that you're accidentally deleting checks that don't exist anywhere else other than incidentally here. So it's less clear what's important without thinking it through each time. now all that to say I don't think it's strictly that big of a deal either way - there's always tons of room for taste that can make one or the other way better in practice. But that was my gut-reaction when reading your comment.

skydhash 2mo ago

> Contra Kent and, it seems, prevailing wisdom, I think simply deleting test1 is by far the simplest, clearest and best way. Provided that your testing framework tells you which specific assertion failed (e.g., by telling you the line number in a stack trace), you do know exactly where the problem is. The issue with your approach is that codepath execution is more of a graph than a linear timeline. With one test, you artificially constrains it to a linear timeline even if the assertions are correct. With multiples and independent you only assert one specific node. That lets you switch up how you do the preliminary steps. I much prefer an exhaustive test unit for step 1, and just a regular call in step 2 and step 3.

akoboldfrying 2mo ago
If you need test1's logic as setup for test2, then you already have that "linear timeline" -- for test2 all by itself. Additionally (that is, redundantly) running the same setup code "prefix" by itself in test1 doesn't remove test2's linear timeline. ETA: I'm assuming your objection to a "linear timeline" is that it reduces the potential for running tests in parallel -- have I got that right? If not, what do you see as being the problem with it?
- skydhash 2mo ago
  If I know that step 1 is correct for all the combinations (N) of its input (in test 1), in test 2, I only need to test the specific combinations (M) of step 2, treating step 1 as an axiom. So no need to have NxM in one test, or NxM tests. Like if you were testing a drone stabilization software, testing the flying state can always assume that it has indeed taken off. No need to validate that it has done so for each scenarios, I can directly put it the correct value. It’s a contrived example, but that axiomatic aspect helps greatly when designing interfaces to reduce coupling between step 1 and step 2.

dack 2mo ago

I think this could lead to the tests getting harder to reason about over time because individual test size will just grow (many iterations of deleting the simple test in favor of a more complicated test that also asserts the simple things) and if you start simplifying the more complicated test later, you may not realize that you're accidentally deleting checks that don't exist anywhere else other than incidentally here. So it's less clear what's important without thinking it through each time. now all that to say I don't think it's strictly that big of a deal either way - there's always tons of room for taste that can make one or the other way better in practice. But that was my gut-reaction when reading your comment.