3 ms·

> testing Sounds like I should give !rspec a go... Since the rest of your comment was super informative, I'm interested in your take on RSpec and Cucumber? >

> testing Sounds like I should give !rspec a go... Since the rest of your comment was super informative, I'm interested in your take on RSpec and Cucumber? > RBS+Steep Hot damn I am out of date and this sounds amazing. Can't wait to try it out... Thank you for posting! > not too sure what you're referring to Best guess is it's like with patterns: Ruby doesn't have that many "patterns" (particularly the canonical kind) because they're just plain unnecessary. Similarly, thinking over other languages... there's just not much that's needed, which might then look like "not a lot".

ncphillips 3y ago
Rspec/cucumber are great, but I’ve recently switched my app to minitest. It’s just as good for unit testing, and its system tests are a joy to work with. The setup and maintenance of rspec/cucumber isn’t worth it for me. It also makes it much easier for me to start new rails projects and get to my ideal workflow, because it’s the default workflow.
calineczka 3y ago
> Cucumber In every project that I've seen using it, it was a tremendous unnecessary waste of time in comparison to writing feature/acceptance tests just in Ruby. Gherkin is just such a poor language in terms of expressiveness compared to Ruby. Not to mention things like poor REPL experience when writing the scenarios.
lloeki 3y ago
> I'm interested in your take on RSpec and Cucumber? I found that Cucumber misses a bit on its promise: in theory things are perfectly described by high level behaviours but in practice it's a bit leaky and I regularly had to work around the tool, which was of disservice to readability. After all if you are faced with a high level descriptive test but have to know about some internals as to why this or that case has to be tested or why it has to be written this or that way (sometimes tacky ways) then it's probably that the tool is not quite fit. Rspec (or minitest/spec) eschews that: essentially it's just Ruby so you write what you need. The other side of the coin is that things you write might not be as "high level" as you initially want: you can have something working but very raw and possibly a tangled hard to maintain mess when it should be "higher level" (or several levels composed together). IOW the big advantage of Rspec is that it scales across from "lowest level" unit testing to "highest level" behaviour so it's always fit, but then it's on you to pick the "proper" level that fits and design your specs accordingly. An example (not perfect by any means but gives a good idea of composition), here we extracted a bunch of functional ("integrated") behaviours into separate examples: https://github.com/DataDog/dd-trace-rb/blob/57976d52bde573f9ea65cdba6a7fee6da290f13a/spec/datadog/appsec/contrib/support/integration/shared_examples.rb https://github.com/DataDog/dd-trace-rb/blob/57976d52bde573f9... Note that some are nested, and with a flick they test behaviour when e.g `appsec` is enabled or `tracing` is disabled. This creates a ton of combinations for only a few tests and coverage goes through the roof. Now here's how we use that: https://github.com/DataDog/dd-trace-rb/blob/57976d52bde573f9ea65cdba6a7fee6da290f13a/spec/datadog/appsec/contrib/rack/integration_test_spec.rb#L225 https://github.com/DataDog/dd-trace-rb/blob/57976d52bde573f9... Notice that magically every case is going to be tested and so we're absolutely sure that it works whatever a user toggles in their configuration. Notice that we create a Rack app, and then the tests proceed, just picking whatever app is in that `let :app`, which leads me to something we have yet to do: https://github.com/DataDog/dd-trace-rb/blob/57976d52bde573f9ea65cdba6a7fee6da290f13a/spec/datadog/appsec/contrib/rails/integration_test_spec.rb#L174 https://github.com/DataDog/dd-trace-rb/blob/57976d52bde573f9... https://github.com/DataDog/dd-trace-rb/blob/57976d52bde573f9ea65cdba6a7fee6da290f13a/spec/datadog/appsec/contrib/sinatra/integration_test_spec.rb#L178 https://github.com/DataDog/dd-trace-rb/blob/57976d52bde573f9... Notice how essentially just the `let :app` setup differs, but tests are entirely similar to the Rack ones (IIRC save for a couple that don't apply to Rack) so actually the overall behaviour can be entirely extracted and shared too! The result would be a per-framework test suite would be a) set up :app for that specific framework and b) a single `it_behaves_like "my super high level behaviour"` line, shared across all frameworks. Bonus (not sure you're familiar with it so here goes): this is tested using Rack::Test, which thanks to Rack's design provides a bunch of request methods (`get` and so on) that merely call the Rack stack as if it were a web server. IOW the whole app code is called just as if there were say Puma or Webrick in front but without ever creating a web server or a socket or whatnot, so it's stupidly fast, and since we're straight in the Ruby code we can mock or stub or allow/expect methods to be received or whatever is needed to our heart's content. Every bit conspires to produce a simple, readable, compact test suite that combinatorially covers every case and runs in a stupidly short amount of time.
- RangerScience 3y ago
  Interesting! This is definitely good food-for-thought, thank you! (At current gig, seems even more clear that we're not making good uses of contexts/etc; both in the sense that we're not using them, and that when we use them we're not using them like this). It does seem like this can cover a bunch of what I usually turn to Cucumber for - the higher-level descriptions - although I think it reinforces how I generally like to use it: for the multi-step full user flow integration "tests" (leaning heavily towards the "this is a product description" and away from "these should be comprehensive"). That is: use something like RSpec to get your ~100% test coverage via unit tests, and then use Cucumber to both (1) ease communication with what a feature does, and (2) exercise all the code involved in a main user path. TL;DR - Seems like the happiest path is something like (1) unit tests that barely use contexts, (2) integration (request, controller) tests that heavily use contexts (your examples), (2) feature (cucumber) tests for multi-step feature flows. Unit tests should aim for 100% coverage (but it's OK to miss); integration tests should aim to cover 100% of the different kinds of situations that are encountered (; feature tests should aim to communicate what users actually do. PS - haha, if you're still at Datadog, talking with a colleague of yours on the sales side about switching from New Relic to y'all. Knowing this quality of code is under there is a a definite point in your favor :)
  - lloeki 3y ago
    Thing is, once you have 1) and 2), the added complexity of bringing in, integrating, and writing for a different tool to achieve 3) begins to make little sense, when you can just go along and do it just as well in rspec anyway... It's a matter of balance and heavily depends on the project. > if you're still at Datadog As a matter of fact I am: curl -s https://github.com/DataDog/dd-trace-rb/commit/176c642ca73679cabc5fa1a113bc9b600aa04dcd.patch | grep '^From:' Shoot me an email if you feel like chatting.

ncphillips 3y ago

Rspec/cucumber are great, but I’ve recently switched my app to minitest. It’s just as good for unit testing, and its system tests are a joy to work with. The setup and maintenance of rspec/cucumber isn’t worth it for me. It also makes it much easier for me to start new rails projects and get to my ideal workflow, because it’s the default workflow.

calineczka 3y ago

> Cucumber In every project that I've seen using it, it was a tremendous unnecessary waste of time in comparison to writing feature/acceptance tests just in Ruby. Gherkin is just such a poor language in terms of expressiveness compared to Ruby. Not to mention things like poor REPL experience when writing the scenarios.

lloeki 3y ago

> I'm interested in your take on RSpec and Cucumber? I found that Cucumber misses a bit on its promise: in theory things are perfectly described by high level behaviours but in practice it's a bit leaky and I regularly had to work around the tool, which was of disservice to readability. After all if you are faced with a high level descriptive test but have to know about some internals as to why this or that case has to be tested or why it has to be written this or that way (sometimes tacky ways) then it's probably that the tool is not quite fit. Rspec (or minitest/spec) eschews that: essentially it's just Ruby so you write what you need. The other side of the coin is that things you write might not be as "high level" as you initially want: you can have something working but very raw and possibly a tangled hard to maintain mess when it should be "higher level" (or several levels composed together). IOW the big advantage of Rspec is that it scales across from "lowest level" unit testing to "highest level" behaviour so it's always fit, but then it's on you to pick the "proper" level that fits and design your specs accordingly. An example (not perfect by any means but gives a good idea of composition), here we extracted a bunch of functional ("integrated") behaviours into separate examples: https://github.com/DataDog/dd-trace-rb/blob/57976d52bde573f9ea65cdba6a7fee6da290f13a/spec/datadog/appsec/contrib/support/integration/shared_examples.rb https://github.com/DataDog/dd-trace-rb/blob/57976d52bde573f9... Note that some are nested, and with a flick they test behaviour when e.g `appsec` is enabled or `tracing` is disabled. This creates a ton of combinations for only a few tests and coverage goes through the roof. Now here's how we use that: https://github.com/DataDog/dd-trace-rb/blob/57976d52bde573f9ea65cdba6a7fee6da290f13a/spec/datadog/appsec/contrib/rack/integration_test_spec.rb#L225 https://github.com/DataDog/dd-trace-rb/blob/57976d52bde573f9... Notice that magically every case is going to be tested and so we're absolutely sure that it works whatever a user toggles in their configuration. Notice that we create a Rack app, and then the tests proceed, just picking whatever app is in that `let :app`, which leads me to something we have yet to do: https://github.com/DataDog/dd-trace-rb/blob/57976d52bde573f9ea65cdba6a7fee6da290f13a/spec/datadog/appsec/contrib/rails/integration_test_spec.rb#L174 https://github.com/DataDog/dd-trace-rb/blob/57976d52bde573f9... https://github.com/DataDog/dd-trace-rb/blob/57976d52bde573f9ea65cdba6a7fee6da290f13a/spec/datadog/appsec/contrib/sinatra/integration_test_spec.rb#L178 https://github.com/DataDog/dd-trace-rb/blob/57976d52bde573f9... Notice how essentially just the `let :app` setup differs, but tests are entirely similar to the Rack ones (IIRC save for a couple that don't apply to Rack) so actually the overall behaviour can be entirely extracted and shared too! The result would be a per-framework test suite would be a) set up :app for that specific framework and b) a single `it_behaves_like "my super high level behaviour"` line, shared across all frameworks. Bonus (not sure you're familiar with it so here goes): this is tested using Rack::Test, which thanks to Rack's design provides a bunch of request methods (`get` and so on) that merely call the Rack stack as if it were a web server. IOW the whole app code is called just as if there were say Puma or Webrick in front but without ever creating a web server or a socket or whatnot, so it's stupidly fast, and since we're straight in the Ruby code we can mock or stub or allow/expect methods to be received or whatever is needed to our heart's content. Every bit conspires to produce a simple, readable, compact test suite that combinatorially covers every case and runs in a stupidly short amount of time.

RangerScience 3y ago
Interesting! This is definitely good food-for-thought, thank you! (At current gig, seems even more clear that we're not making good uses of contexts/etc; both in the sense that we're not using them, and that when we use them we're not using them like this). It does seem like this can cover a bunch of what I usually turn to Cucumber for - the higher-level descriptions - although I think it reinforces how I generally like to use it: for the multi-step full user flow integration "tests" (leaning heavily towards the "this is a product description" and away from "these should be comprehensive"). That is: use something like RSpec to get your ~100% test coverage via unit tests, and then use Cucumber to both (1) ease communication with what a feature does, and (2) exercise all the code involved in a main user path. TL;DR - Seems like the happiest path is something like (1) unit tests that barely use contexts, (2) integration (request, controller) tests that heavily use contexts (your examples), (2) feature (cucumber) tests for multi-step feature flows. Unit tests should aim for 100% coverage (but it's OK to miss); integration tests should aim to cover 100% of the different kinds of situations that are encountered (; feature tests should aim to communicate what users actually do. PS - haha, if you're still at Datadog, talking with a colleague of yours on the sales side about switching from New Relic to y'all. Knowing this quality of code is under there is a a definite point in your favor :)
- lloeki 3y ago
  Thing is, once you have 1) and 2), the added complexity of bringing in, integrating, and writing for a different tool to achieve 3) begins to make little sense, when you can just go along and do it just as well in rspec anyway... It's a matter of balance and heavily depends on the project. > if you're still at Datadog As a matter of fact I am: curl -s https://github.com/DataDog/dd-trace-rb/commit/176c642ca73679cabc5fa1a113bc9b600aa04dcd.patch | grep '^From:' Shoot me an email if you feel like chatting.