3 ms·

This is a great post and I agree with almost everything in it 100% except for this part: > And—yes—you can read this as my being unlikely to use Rails the next

This is a great post and I agree with almost everything in it 100% except for this part: > And—yes—you can read this as my being unlikely to use Rails the next time I'm building a traditional multi-page application. Really? The way I do it is I write a client library for the API and run the tests in Rspec on top of the client library. Sure I'm going through the whole stack, but the thing operates as fast as you can serve normal users! Surely your large project can handle 5000 concurrent requests? While that might seem like a lot of buildout (scalable testing servers), I'd argue the opposite is a lot of buildout: writing an application without Rails. Granted my approach might be most feasible for Rails API + Ember, but I really don't see the speed of my tests passing as what's holding me back. The other thing that is nice is that once you start needing to speed up endpoints by pulling them into a compiled language, you can keep your tests! Everything is going through the API anyway! So feel free to Nim or Rust the CPU heavy endpoint or Redis-cache the write-once-in-a-while-and-keep endpoint.

edwinnathaniel 11y ago
It will hurt when your end-to-end tests (minus UI) must be run per check-in and as your code gets bigger. Some companies invest in huge infrastructure to speed things up (paralleling until you can't...) but in that situation, suddenly Devs can no longer build locally without biting their tongue. Setting up end-to-end test "per test-case" is also tedious (should user be logged-in in order for this test-case to run? what should be the state before the tests are run and what should be the state after the tests are run? what if the state of the system have the expected _and_ more e.g.: side-effect?)
- 3pt14159 11y ago
  > Should user be logged-in in order for this test-case to run? Why not? This is a simple helper function or easily parallelizable block. Same with creating a sub-resource before modifying it. People act like this is the end of the world, but it isn't. And if you like, you can still do your (in my controversial opinion) silly unit tests and you can still parallelize those too! Whether a dev can build a project locally or not does not relate to whether or not the tests can be run in parallel.
  - edwinnathaniel 11y ago
    Oh, for sure, it's a simple helper function but the ramification is huge! 1. Which user? which role should the test written as? 2. Suddenly bug showed up because expectation between roles differ (usually the bug is in the test framework) No, it's not the end of the world, just programmer's productivity go down...people can still work for the project just isn't fast as it used to be anymore...
    - 3pt14159 11y ago
      Compared to unit tests where the concept of the user doesn't even exist or needs to be shimmed in with a proxy object, I really don't think the user differences in ACL are the deciding factor between the two. As for programmer productivity, proper parallelization of these tests results in sub second test suite runtime. Maybe more if there is a new dependency that the testing servers need to install, but it really isn't that bad.
- crdoconnor 11y ago
  >It will hurt when your end-to-end tests (minus UI) must be run per check-in and as your code gets bigger. > >Some companies invest in huge infrastructure to speed things up (paralleling until you can't...) but in that situation, suddenly Devs can no longer build locally without biting their tongue. That's why you pick a few smoke tests to run before checking the code in and let the full regression test suite run while you're doing other stuff. >Setting up end-to-end test "per test-case" is also tedious (should user be logged-in in order for this test-case to run? what should be the state before the tests are run and what should be the state after the tests are run? what if the state of the system have the expected _and_ more e.g.: side-effect?) You need a decent integration testing framework to help here.

edwinnathaniel 11y ago

It will hurt when your end-to-end tests (minus UI) must be run per check-in and as your code gets bigger. Some companies invest in huge infrastructure to speed things up (paralleling until you can't...) but in that situation, suddenly Devs can no longer build locally without biting their tongue. Setting up end-to-end test "per test-case" is also tedious (should user be logged-in in order for this test-case to run? what should be the state before the tests are run and what should be the state after the tests are run? what if the state of the system have the expected _and_ more e.g.: side-effect?)

3pt14159 11y ago
> Should user be logged-in in order for this test-case to run? Why not? This is a simple helper function or easily parallelizable block. Same with creating a sub-resource before modifying it. People act like this is the end of the world, but it isn't. And if you like, you can still do your (in my controversial opinion) silly unit tests and you can still parallelize those too! Whether a dev can build a project locally or not does not relate to whether or not the tests can be run in parallel.
- edwinnathaniel 11y ago
  Oh, for sure, it's a simple helper function but the ramification is huge! 1. Which user? which role should the test written as? 2. Suddenly bug showed up because expectation between roles differ (usually the bug is in the test framework) No, it's not the end of the world, just programmer's productivity go down...people can still work for the project just isn't fast as it used to be anymore...
  - 3pt14159 11y ago
    Compared to unit tests where the concept of the user doesn't even exist or needs to be shimmed in with a proxy object, I really don't think the user differences in ACL are the deciding factor between the two. As for programmer productivity, proper parallelization of these tests results in sub second test suite runtime. Maybe more if there is a new dependency that the testing servers need to install, but it really isn't that bad.
crdoconnor 11y ago
>It will hurt when your end-to-end tests (minus UI) must be run per check-in and as your code gets bigger. > >Some companies invest in huge infrastructure to speed things up (paralleling until you can't...) but in that situation, suddenly Devs can no longer build locally without biting their tongue. That's why you pick a few smoke tests to run before checking the code in and let the full regression test suite run while you're doing other stuff. >Setting up end-to-end test "per test-case" is also tedious (should user be logged-in in order for this test-case to run? what should be the state before the tests are run and what should be the state after the tests are run? what if the state of the system have the expected _and_ more e.g.: side-effect?) You need a decent integration testing framework to help here.