3 ms·

It will hurt when your end-to-end tests (minus UI) must be run per check-in and as your code gets bigger. Some companies invest in huge infrastructure to speed

It will hurt when your end-to-end tests (minus UI) must be run per check-in and as your code gets bigger. Some companies invest in huge infrastructure to speed things up (paralleling until you can't...) but in that situation, suddenly Devs can no longer build locally without biting their tongue. Setting up end-to-end test "per test-case" is also tedious (should user be logged-in in order for this test-case to run? what should be the state before the tests are run and what should be the state after the tests are run? what if the state of the system have the expected _and_ more e.g.: side-effect?)

3pt14159 11y ago
> Should user be logged-in in order for this test-case to run? Why not? This is a simple helper function or easily parallelizable block. Same with creating a sub-resource before modifying it. People act like this is the end of the world, but it isn't. And if you like, you can still do your (in my controversial opinion) silly unit tests and you can still parallelize those too! Whether a dev can build a project locally or not does not relate to whether or not the tests can be run in parallel.
- edwinnathaniel 11y ago
  Oh, for sure, it's a simple helper function but the ramification is huge! 1. Which user? which role should the test written as? 2. Suddenly bug showed up because expectation between roles differ (usually the bug is in the test framework) No, it's not the end of the world, just programmer's productivity go down...people can still work for the project just isn't fast as it used to be anymore...
  - 3pt14159 11y ago
    Compared to unit tests where the concept of the user doesn't even exist or needs to be shimmed in with a proxy object, I really don't think the user differences in ACL are the deciding factor between the two. As for programmer productivity, proper parallelization of these tests results in sub second test suite runtime. Maybe more if there is a new dependency that the testing servers need to install, but it really isn't that bad.
crdoconnor 11y ago
>It will hurt when your end-to-end tests (minus UI) must be run per check-in and as your code gets bigger. > >Some companies invest in huge infrastructure to speed things up (paralleling until you can't...) but in that situation, suddenly Devs can no longer build locally without biting their tongue. That's why you pick a few smoke tests to run before checking the code in and let the full regression test suite run while you're doing other stuff. >Setting up end-to-end test "per test-case" is also tedious (should user be logged-in in order for this test-case to run? what should be the state before the tests are run and what should be the state after the tests are run? what if the state of the system have the expected _and_ more e.g.: side-effect?) You need a decent integration testing framework to help here.

3pt14159 11y ago

> Should user be logged-in in order for this test-case to run? Why not? This is a simple helper function or easily parallelizable block. Same with creating a sub-resource before modifying it. People act like this is the end of the world, but it isn't. And if you like, you can still do your (in my controversial opinion) silly unit tests and you can still parallelize those too! Whether a dev can build a project locally or not does not relate to whether or not the tests can be run in parallel.

edwinnathaniel 11y ago
Oh, for sure, it's a simple helper function but the ramification is huge! 1. Which user? which role should the test written as? 2. Suddenly bug showed up because expectation between roles differ (usually the bug is in the test framework) No, it's not the end of the world, just programmer's productivity go down...people can still work for the project just isn't fast as it used to be anymore...
- 3pt14159 11y ago
  Compared to unit tests where the concept of the user doesn't even exist or needs to be shimmed in with a proxy object, I really don't think the user differences in ACL are the deciding factor between the two. As for programmer productivity, proper parallelization of these tests results in sub second test suite runtime. Maybe more if there is a new dependency that the testing servers need to install, but it really isn't that bad.

crdoconnor 11y ago

>It will hurt when your end-to-end tests (minus UI) must be run per check-in and as your code gets bigger. > >Some companies invest in huge infrastructure to speed things up (paralleling until you can't...) but in that situation, suddenly Devs can no longer build locally without biting their tongue. That's why you pick a few smoke tests to run before checking the code in and let the full regression test suite run while you're doing other stuff. >Setting up end-to-end test "per test-case" is also tedious (should user be logged-in in order for this test-case to run? what should be the state before the tests are run and what should be the state after the tests are run? what if the state of the system have the expected _and_ more e.g.: side-effect?) You need a decent integration testing framework to help here.