3 ms·

An assert message says what went wrong, and on which code line. How on earth does it help to make just one? The arrange part might take seconds for a nontrivial

An assert message says what went wrong, and on which code line. How on earth does it help to make just one? The arrange part might take seconds for a nontrivial test and that would need to be duplicated both in code and execution time to make two asserts. If you painstakingly craft a scenario where you create a rectangle of a specific expected size why wouldn’t it be acceptable to assert both the width and height of the the rectangle after you have created it? assert_equal(20, w, … assert_equal(10, h, … A dogmatic rule would just lead to an objectively worse test where you assert an expression containing both width and height in a single assert? assert_true(w == 20 && h == 10,…) So I can only assume the rule also prohibits any compound/Boolean expressions in the asserts then? Otherwise you can just combine any number of asserts into one (including mutating state within the expression itself to emulate multiple asserts with mutation between)!

TeMPOraL 4y ago
> So I can only assume the rule also prohibits any compound/Boolean expressions in the asserts then? Otherwise you can just combine any number of asserts into one That's what's bound to happen under that rule. People just start writing their complex tests in helper functions, and then write assert_true(myTotallySimpleAndFocusedTestOn(result))
jeremyjh 4y ago
Apparently this is yet another “best practice” preached by people using crappy tools.
ljm 4y ago
I’ve seen people take a dogmatic approach to this in Ruby without really applying any critical thought, because one assertion per test means your test is ‘clean’. The part that is glossed over is that the test suite takes several hours to run on your machine, so you delegate it to a CI pipeline and then fork out for parallel execution (pun intended) and complex layers of caching so your suite takes 15 minutes rather than 2 and a half hours. It’s monumentally wasteful and the tests aren’t any easier to follow because of it. The suite doesn’t have to be that slow, but it’s inevitable when every single assertion requires application state to be rebuilt from scratch, even when no state is expected to change between assertions, especially when you’re just doing assertions like ‘assert http status is 201’ and ‘assert response body is someJson’.
- hamandcheese 4y ago
  > I’ve seen people take a dogmatic approach to this in Ruby I came up in ruby, heard this, and quickly decided it was stupid.
- mollerhoj 4y ago
  Yes, you got us rubyists there. :-( Its the unfortunate result of trying to avoid premature optimization and strive for clarity instead. Something thats usually sound advice. Enginnering decisions have tradeoffs. When the testsuite becomes too slow, it might be time to reconsider those tradeoffs. Usually though, I find that to road to fast tests is to reduce/remove slow things (almost always some form of IO) not to combine 10 small tests into one big.
  - ljm 4y ago
    I think it’s a sound strategy more often than not, it’s just that RSpec’s DSL can make those trade-offs unclear, especially if you use Rubocop and follow its default RSpec rules. It just so happens that your tests become IO bound because those small tests in aggregate hammer your DB and the network purely to set up state. So if you only do it once by being more deliberate with your tests, you’re in a better place.
  - int_19h 4y ago
    I'd argue that it's the unfortunate result of Ruby being at the center of the Agile and XP scene back when it first became prominent (the manifesto etc) - because that scene is also where the more cultish varieties of TDD originated.
- OJFord 4y ago
  > I’ve seen people take a dogmatic approach to this in Ruby without really applying any critical thought, because one assertion per test means your test is ‘clean’. I can't speak for Ruby, but what I would call 'clean' and happily dogmatise is that assertions should come at the end, after setup and exercise. I don't care how many there are, but they come last. I really hate tests that look like: setup() exercise(but_not_for_the_last_time) assert state.currently = blah state.do_other() something(state) assert state.now = blegh And so on. It stinks of an integration test forced into a unit testing framework. I like them to look like: foo = FooFactory(prop="whatever") result = do(foo) assert result == "bar" I.e. some setup, something clearly under test, and then the assertion(s) checking the result.
flavius29663 4y ago
That can be considered one logical assertion though. You're asserting the size of the rectangle. You can even extract an assert helper function AssertRectangleSize(20,10)
- alkonaut 4y ago
  Exactly. But if I assert N properties of an object is that then one assert logically for any N? At what point does it stop? Applying any rule dogmatically is often bad, and this is no exception. The is that we don’t like lacking rules. Especially it goes to hell when people start adding code analysis to enforce it, and then developers start writing poor code that passes the analysis. One assert imo isn’t even a good starting point that might need occasional exceptions.
  - flavius29663 4y ago
    It shouldn't be dogmatic, but I think it should be something to think about when writing the test or reviewing one. A test should be single responsibility too, in order for it to not be brittle. I disagree, it can be a good starting point for most cases. You should be able to condense your test in 3 steps (arrange, act, assert) each one a single line, and even if you do not do it because setup is too complicated, it's not worth it for that single test, you assert more things etc., I think the mental exercise to try to think: "can this be made in a 3 line test" is invaluable in writing maintainable tests. > is that then one assert logically for any N? At what point does it stop? This is one of the hard things about good tests, they are a little bit of art too. Maybe you can apply the single responsibility like I said before: the test should change for one reason only. By one reason I mean one "person/role": it should change if the CFO of our clients want something differet, or if Mark from IT wants some change. I am not stressing or enforcing single asserts too much, I feel like tests allow a little bit of leeway in many ways, as long as the decision enhances expressiveness. If the extra lines are not making the test clearer, if a single assert would be clearer for the story that the test is telling, then it should go into a single assert. If I can break the story into multiple stories that still make sense, then I do that, such that each story has it's own strong storyline.
ImPleadThe5th 4y ago
The way it's been explained to me is that because one assert failing blocks the other assertions from running you don't get a "full" picture of what went wrong. So instead of: - error W doesn't equal 20 Fix that Run test again - error H doesn't equal 10 Fix that Run test again It's - Error Width doesn't equal 20 - Error Height doesn't equal 10 Fix both Run test I think the time savings are negligible though. And it makes testing even more tedious, as if people needed any additional reasons to avoid writing tests.
hinkley 4y ago
Only a few programming languages have a facility to render that second assertion in a human readable way (python surprised me with this). Most C influenced languages will just present you with “assertion failed” or “expected true to be false” which means nothing. Test failure messages should be actionable, and that action is not, “read the test to see what went wrong”.

TeMPOraL 4y ago

> So I can only assume the rule also prohibits any compound/Boolean expressions in the asserts then? Otherwise you can just combine any number of asserts into one That's what's bound to happen under that rule. People just start writing their complex tests in helper functions, and then write assert_true(myTotallySimpleAndFocusedTestOn(result))

jeremyjh 4y ago

Apparently this is yet another “best practice” preached by people using crappy tools.

ljm 4y ago

I’ve seen people take a dogmatic approach to this in Ruby without really applying any critical thought, because one assertion per test means your test is ‘clean’. The part that is glossed over is that the test suite takes several hours to run on your machine, so you delegate it to a CI pipeline and then fork out for parallel execution (pun intended) and complex layers of caching so your suite takes 15 minutes rather than 2 and a half hours. It’s monumentally wasteful and the tests aren’t any easier to follow because of it. The suite doesn’t have to be that slow, but it’s inevitable when every single assertion requires application state to be rebuilt from scratch, even when no state is expected to change between assertions, especially when you’re just doing assertions like ‘assert http status is 201’ and ‘assert response body is someJson’.

hamandcheese 4y ago
> I’ve seen people take a dogmatic approach to this in Ruby I came up in ruby, heard this, and quickly decided it was stupid.
mollerhoj 4y ago
Yes, you got us rubyists there. :-( Its the unfortunate result of trying to avoid premature optimization and strive for clarity instead. Something thats usually sound advice. Enginnering decisions have tradeoffs. When the testsuite becomes too slow, it might be time to reconsider those tradeoffs. Usually though, I find that to road to fast tests is to reduce/remove slow things (almost always some form of IO) not to combine 10 small tests into one big.
- ljm 4y ago
  I think it’s a sound strategy more often than not, it’s just that RSpec’s DSL can make those trade-offs unclear, especially if you use Rubocop and follow its default RSpec rules. It just so happens that your tests become IO bound because those small tests in aggregate hammer your DB and the network purely to set up state. So if you only do it once by being more deliberate with your tests, you’re in a better place.
- int_19h 4y ago
  I'd argue that it's the unfortunate result of Ruby being at the center of the Agile and XP scene back when it first became prominent (the manifesto etc) - because that scene is also where the more cultish varieties of TDD originated.
OJFord 4y ago
> I’ve seen people take a dogmatic approach to this in Ruby without really applying any critical thought, because one assertion per test means your test is ‘clean’. I can't speak for Ruby, but what I would call 'clean' and happily dogmatise is that assertions should come at the end, after setup and exercise. I don't care how many there are, but they come last. I really hate tests that look like: setup() exercise(but_not_for_the_last_time) assert state.currently = blah state.do_other() something(state) assert state.now = blegh And so on. It stinks of an integration test forced into a unit testing framework. I like them to look like: foo = FooFactory(prop="whatever") result = do(foo) assert result == "bar" I.e. some setup, something clearly under test, and then the assertion(s) checking the result.

flavius29663 4y ago

That can be considered one logical assertion though. You're asserting the size of the rectangle. You can even extract an assert helper function AssertRectangleSize(20,10)

alkonaut 4y ago
Exactly. But if I assert N properties of an object is that then one assert logically for any N? At what point does it stop? Applying any rule dogmatically is often bad, and this is no exception. The is that we don’t like lacking rules. Especially it goes to hell when people start adding code analysis to enforce it, and then developers start writing poor code that passes the analysis. One assert imo isn’t even a good starting point that might need occasional exceptions.
- flavius29663 4y ago
  It shouldn't be dogmatic, but I think it should be something to think about when writing the test or reviewing one. A test should be single responsibility too, in order for it to not be brittle. I disagree, it can be a good starting point for most cases. You should be able to condense your test in 3 steps (arrange, act, assert) each one a single line, and even if you do not do it because setup is too complicated, it's not worth it for that single test, you assert more things etc., I think the mental exercise to try to think: "can this be made in a 3 line test" is invaluable in writing maintainable tests. > is that then one assert logically for any N? At what point does it stop? This is one of the hard things about good tests, they are a little bit of art too. Maybe you can apply the single responsibility like I said before: the test should change for one reason only. By one reason I mean one "person/role": it should change if the CFO of our clients want something differet, or if Mark from IT wants some change. I am not stressing or enforcing single asserts too much, I feel like tests allow a little bit of leeway in many ways, as long as the decision enhances expressiveness. If the extra lines are not making the test clearer, if a single assert would be clearer for the story that the test is telling, then it should go into a single assert. If I can break the story into multiple stories that still make sense, then I do that, such that each story has it's own strong storyline.

ImPleadThe5th 4y ago

The way it's been explained to me is that because one assert failing blocks the other assertions from running you don't get a "full" picture of what went wrong. So instead of: - error W doesn't equal 20 Fix that Run test again - error H doesn't equal 10 Fix that Run test again It's - Error Width doesn't equal 20 - Error Height doesn't equal 10 Fix both Run test I think the time savings are negligible though. And it makes testing even more tedious, as if people needed any additional reasons to avoid writing tests.

hinkley 4y ago

Only a few programming languages have a facility to render that second assertion in a human readable way (python surprised me with this). Most C influenced languages will just present you with “assertion failed” or “expected true to be false” which means nothing. Test failure messages should be actionable, and that action is not, “read the test to see what went wrong”.