5 ms·

A whole domain for a rather minor point like this, which is not even all that correct in the first place? All the arguments in this article promise very minor

by dpc_pw 6y ago

A whole domain for a rather minor point like this, which is not even all that correct in the first place? All the arguments in this article promise very minor benefits. > When your test breaks, by fail or error, further assertions are never executed, and test coverage is reduced. Oh wow. I wish I had your problems. :D Sure, a failing assertion can hide a bigger problem being there. But most of the time if one assertion fails, rest of the code is useless anyway, and will just generate noise. You could design assertions in a very sophisticated way to minimize that problem, and optimize amount of information... but that's complex and brittle and time consuming. The whole idea seems like marginal return optimization that takes way too much effort to be worth it. If you have a decent test coverage with well written tests ... you're golden and you probably have more important things to do than trying to tweak your tests to optimize your assertions just in case something sometimes fails during development work.

r0s 6y ago
> You could design assertions in a very sophisticated way to minimize that problem, and optimize amount of information... but that's complex and brittle and time consuming. My point here is this optimization is really pretty simple. One target state per test, and soft assertions for the rest. It's actually a simplification rather than adding complexity. Soft asserts make your tests more robust by definition, lots of halting assertions break more often, they are brittle by design. > If you have a decent test coverage with well written tests ... you're golden Hey if your tests catch all bugs, it ain't broke, don't fix it! Totally understandable if you aren't looking for optimizations like this. For the massive test suites I work on, it's necessary to at least consider these concepts.
contravariant 6y ago
>You could design assertions in a very sophisticated way to minimize that problem, [...] This sounds like the exact opposite way you need to design assertions. If code can continue as normal when an assertion fails then why put the assertion in there anyway?
AlexanderNull 6y ago
> most of the time if one assertion fails, rest of the code is useless anyway You either don't write tests or you're already writing them in the right way (sounds like the later). I've seen my fair share of what I would consider compound tests that have multiple asserts in tests that would crash execution of that test even though 3 lines down in that same test is a completely different bit of state being tested. This is hopefully less of an issue in unit tests but my gosh I've seen it way too much in integration tests. It can get worse still when one of these initial assertions starts failing, a lazy dev goes in to address the problem, finds that one assertion isn't an issue worth addressing for now, labels the whole test as a KnownIssue and moves on leaving us at risk for the other issues covered in the later asserts to break without warning at a later point in time! (only seen this twice luckily)

by dpc_pw 6y ago

r0s 6y ago

> You could design assertions in a very sophisticated way to minimize that problem, and optimize amount of information... but that's complex and brittle and time consuming. My point here is this optimization is really pretty simple. One target state per test, and soft assertions for the rest. It's actually a simplification rather than adding complexity. Soft asserts make your tests more robust by definition, lots of halting assertions break more often, they are brittle by design. > If you have a decent test coverage with well written tests ... you're golden Hey if your tests catch all bugs, it ain't broke, don't fix it! Totally understandable if you aren't looking for optimizations like this. For the massive test suites I work on, it's necessary to at least consider these concepts.

contravariant 6y ago

>You could design assertions in a very sophisticated way to minimize that problem, [...] This sounds like the exact opposite way you need to design assertions. If code can continue as normal when an assertion fails then why put the assertion in there anyway?

AlexanderNull 6y ago

> most of the time if one assertion fails, rest of the code is useless anyway You either don't write tests or you're already writing them in the right way (sounds like the later). I've seen my fair share of what I would consider compound tests that have multiple asserts in tests that would crash execution of that test even though 3 lines down in that same test is a completely different bit of state being tested. This is hopefully less of an issue in unit tests but my gosh I've seen it way too much in integration tests. It can get worse still when one of these initial assertions starts failing, a lazy dev goes in to address the problem, finds that one assertion isn't an issue worth addressing for now, labels the whole test as a KnownIssue and moves on leaving us at risk for the other issues covered in the later asserts to break without warning at a later point in time! (only seen this twice luckily)