3 ms·

> Some unit testing frameworks expect their users to use assertions to do the job, others are very much against it because they want to separate between the fai

> Some unit testing frameworks expect their users to use assertions to do the job, others are very much against it because they want to separate between the failures of the system under test from failures of the test. (If that matters, I'm in the later camp). How useful is this distinction in practice? A failing test is going to examined in detail and that examination is going to reveal whether the system under test failed or if the test itself failed. I guess I am asking, when is this distinction useful?

twhitmore 2mo ago
A typical place this distinction is useful is that it determines whether the test-suite runs to completion with multiple tests, or panics/terminates hard on the first failure.
crabbone 2mo ago
Oh... it's not just useful, people get red in the face and start swinging heavy object in the air when it comes to discussing this. Imagine working in a larger company, where you routinely get close to a hundred of useless emails every day. Imagine they use some garbage mail server like the one provided by Office 365, so that filtering is broken, emails get lost all the time etc. And now your CI is sending you alerts about code breakage, and you need to... wait, not just scroll through an endless Jenkins log, you need to download, unzip the artifacts, figure out which files are the test logs, and from there try to figure out what the test was doing and whether the error has anything to do with your code. This whole process is infuriatingly unnecessary, tedious, it contributes nothing to whatever goals you've set for yourself. It's a toil that you have to engage in every day, perhaps for hours, just to come back with the answer "looks like it's not my problem after all". I know this because I've been on the receiving end of this anger and frustration :) And I've never found a good way to eliminate this problem completely, but I'm sure that narrowing down the number of people the problem is reported to only to the most relevant people helps.

twhitmore 2mo ago

A typical place this distinction is useful is that it determines whether the test-suite runs to completion with multiple tests, or panics/terminates hard on the first failure.

crabbone 2mo ago

Oh... it's not just useful, people get red in the face and start swinging heavy object in the air when it comes to discussing this. Imagine working in a larger company, where you routinely get close to a hundred of useless emails every day. Imagine they use some garbage mail server like the one provided by Office 365, so that filtering is broken, emails get lost all the time etc. And now your CI is sending you alerts about code breakage, and you need to... wait, not just scroll through an endless Jenkins log, you need to download, unzip the artifacts, figure out which files are the test logs, and from there try to figure out what the test was doing and whether the error has anything to do with your code. This whole process is infuriatingly unnecessary, tedious, it contributes nothing to whatever goals you've set for yourself. It's a toil that you have to engage in every day, perhaps for hours, just to come back with the answer "looks like it's not my problem after all". I know this because I've been on the receiving end of this anger and frustration :) And I've never found a good way to eliminate this problem completely, but I'm sure that narrowing down the number of people the problem is reported to only to the most relevant people helps.