29 ms·

I’ve seen people take a dogmatic approach to this in Ruby without really applying any critical thought, because one assertion per test means your test is ‘clean

by ljm 4y ago

I’ve seen people take a dogmatic approach to this in Ruby without really applying any critical thought, because one assertion per test means your test is ‘clean’. The part that is glossed over is that the test suite takes several hours to run on your machine, so you delegate it to a CI pipeline and then fork out for parallel execution (pun intended) and complex layers of caching so your suite takes 15 minutes rather than 2 and a half hours. It’s monumentally wasteful and the tests aren’t any easier to follow because of it. The suite doesn’t have to be that slow, but it’s inevitable when every single assertion requires application state to be rebuilt from scratch, even when no state is expected to change between assertions, especially when you’re just doing assertions like ‘assert http status is 201’ and ‘assert response body is someJson’.

hamandcheese 4y ago
> I’ve seen people take a dogmatic approach to this in Ruby I came up in ruby, heard this, and quickly decided it was stupid.
mollerhoj 4y ago
Yes, you got us rubyists there. :-( Its the unfortunate result of trying to avoid premature optimization and strive for clarity instead. Something thats usually sound advice. Enginnering decisions have tradeoffs. When the testsuite becomes too slow, it might be time to reconsider those tradeoffs. Usually though, I find that to road to fast tests is to reduce/remove slow things (almost always some form of IO) not to combine 10 small tests into one big.
- ljm 4y ago
  I think it’s a sound strategy more often than not, it’s just that RSpec’s DSL can make those trade-offs unclear, especially if you use Rubocop and follow its default RSpec rules. It just so happens that your tests become IO bound because those small tests in aggregate hammer your DB and the network purely to set up state. So if you only do it once by being more deliberate with your tests, you’re in a better place.
- int_19h 4y ago
  I'd argue that it's the unfortunate result of Ruby being at the center of the Agile and XP scene back when it first became prominent (the manifesto etc) - because that scene is also where the more cultish varieties of TDD originated.
OJFord 4y ago
> I’ve seen people take a dogmatic approach to this in Ruby without really applying any critical thought, because one assertion per test means your test is ‘clean’. I can't speak for Ruby, but what I would call 'clean' and happily dogmatise is that assertions should come at the end, after setup and exercise. I don't care how many there are, but they come last. I really hate tests that look like: setup() exercise(but_not_for_the_last_time) assert state.currently = blah state.do_other() something(state) assert state.now = blegh And so on. It stinks of an integration test forced into a unit testing framework. I like them to look like: foo = FooFactory(prop="whatever") result = do(foo) assert result == "bar" I.e. some setup, something clearly under test, and then the assertion(s) checking the result.
- ljm 4y ago
  I think even with integration tests they should still be treated similarly - at the end of the day you are setting expectations on an output given a certain input, there’s just a lot more going on in between. There’s no avoiding it though when you want something end-to-end, or a synthetic test. You’re piling up a whole succession of stateful actions and if you tested them in isolation you would fail to capture bugs that depend on state. In that sense, better to run a ‘signup, authenticate and onboard’ flow in one test instead of breaking it down.
- int_19h 4y ago
  Here's a trivial rewrite that satisfies your dogmatic requirement without any meaningful difference: setup() exercise(but_not_for_the_last_time) was_blah = (state.currently == blah) state.do_other() something(state) assert was_blah && state.now == blegh In fact, this last version is worse, because if do_other() can fail if state wasn't blah, then what you'll get is the exception from that failure interrupting the test before the assert would have been reported.
  - OJFord 4y ago
    Exactly because that's 'without any meaningful difference' is why I don't like that either. I'm not obsessing over purely the 'assert' keyword as you perhaps think I am, it's the structure I don't like.

by ljm 4y ago

hamandcheese 4y ago

> I’ve seen people take a dogmatic approach to this in Ruby I came up in ruby, heard this, and quickly decided it was stupid.

mollerhoj 4y ago

Yes, you got us rubyists there. :-( Its the unfortunate result of trying to avoid premature optimization and strive for clarity instead. Something thats usually sound advice. Enginnering decisions have tradeoffs. When the testsuite becomes too slow, it might be time to reconsider those tradeoffs. Usually though, I find that to road to fast tests is to reduce/remove slow things (almost always some form of IO) not to combine 10 small tests into one big.

ljm 4y ago
I think it’s a sound strategy more often than not, it’s just that RSpec’s DSL can make those trade-offs unclear, especially if you use Rubocop and follow its default RSpec rules. It just so happens that your tests become IO bound because those small tests in aggregate hammer your DB and the network purely to set up state. So if you only do it once by being more deliberate with your tests, you’re in a better place.
int_19h 4y ago
I'd argue that it's the unfortunate result of Ruby being at the center of the Agile and XP scene back when it first became prominent (the manifesto etc) - because that scene is also where the more cultish varieties of TDD originated.

OJFord 4y ago

> I’ve seen people take a dogmatic approach to this in Ruby without really applying any critical thought, because one assertion per test means your test is ‘clean’. I can't speak for Ruby, but what I would call 'clean' and happily dogmatise is that assertions should come at the end, after setup and exercise. I don't care how many there are, but they come last. I really hate tests that look like: setup() exercise(but_not_for_the_last_time) assert state.currently = blah state.do_other() something(state) assert state.now = blegh And so on. It stinks of an integration test forced into a unit testing framework. I like them to look like: foo = FooFactory(prop="whatever") result = do(foo) assert result == "bar" I.e. some setup, something clearly under test, and then the assertion(s) checking the result.

ljm 4y ago
I think even with integration tests they should still be treated similarly - at the end of the day you are setting expectations on an output given a certain input, there’s just a lot more going on in between. There’s no avoiding it though when you want something end-to-end, or a synthetic test. You’re piling up a whole succession of stateful actions and if you tested them in isolation you would fail to capture bugs that depend on state. In that sense, better to run a ‘signup, authenticate and onboard’ flow in one test instead of breaking it down.
int_19h 4y ago
Here's a trivial rewrite that satisfies your dogmatic requirement without any meaningful difference: setup() exercise(but_not_for_the_last_time) was_blah = (state.currently == blah) state.do_other() something(state) assert was_blah && state.now == blegh In fact, this last version is worse, because if do_other() can fail if state wasn't blah, then what you'll get is the exception from that failure interrupting the test before the assert would have been reported.
- OJFord 4y ago
  Exactly because that's 'without any meaningful difference' is why I don't like that either. I'm not obsessing over purely the 'assert' keyword as you perhaps think I am, it's the structure I don't like.