5 ms·

Sure, let's go back to my payroll example. I need to make 8 HTTP requests to properly setup my test from a clean slate, including setting up the deductions, pr

Sure, let's go back to my payroll example. I need to make 8 HTTP requests to properly setup my test from a clean slate, including setting up the deductions, previous earnings, company location and employee home address. If each of those requests takes 50ms, and then I need to make a request to actually execute my test, and verify everything went well, that could easily be 500ms. And that's just what needs to happen to run one end to end test of the payroll calculation feature. It's absolutely valuable to have a few such tests, but I'm not going to run tests for all the weird one off scenarios (like what happens if someone lives in NJ, and works in Yonkers, and the company has a location in NJ as well?) because they'd take minutes to run. I could run that entire test as a unit test in 2ms for the payroll calculation. That lets me run all of the tests I need very quickly.

dnautics 8y ago
can't you run these feature tests in parallel? I'm runnning a system where it full on downloads and mounts containers, and executes them (in a loopback, they get hosted by transient webservers on the test host aka my laptop, but possibly also travis), about 50 of them in parallel with tons of database calls, and it usually takes around 10-20s to complete.
- Macha 8y ago
  It only takes you 20s to start up 50 containers? I doubt I could start 50 alpine containers in that time on my company issue 2015 MBP (Edit: A quick test reveals my personal desktop can in fact accomplish this, but it's much more powerful than my work MBP and has no virtualization overhead since it's running native docker instead of docker for mac). I don't know about jeffasinger's company, but in optimal circumstances, a test instance of our app running against a in memory h2 DB takes 3 minutes to start. The app internally already performs heavy lifting in parallel, so it's also not clear that running multiple instances of the app will make it that much faster...
  - dnautics 8y ago
    Singularity containers, my dev laptop is Linux, way less overhead than docker on mac. I'm simulating request load on a scheduler for compute jobs. In principle those jobs are not generally local, but it helps to stresstest the scheduler.

dnautics 8y ago

can't you run these feature tests in parallel? I'm runnning a system where it full on downloads and mounts containers, and executes them (in a loopback, they get hosted by transient webservers on the test host aka my laptop, but possibly also travis), about 50 of them in parallel with tons of database calls, and it usually takes around 10-20s to complete.

Macha 8y ago
It only takes you 20s to start up 50 containers? I doubt I could start 50 alpine containers in that time on my company issue 2015 MBP (Edit: A quick test reveals my personal desktop can in fact accomplish this, but it's much more powerful than my work MBP and has no virtualization overhead since it's running native docker instead of docker for mac). I don't know about jeffasinger's company, but in optimal circumstances, a test instance of our app running against a in memory h2 DB takes 3 minutes to start. The app internally already performs heavy lifting in parallel, so it's also not clear that running multiple instances of the app will make it that much faster...
- dnautics 8y ago
  Singularity containers, my dev laptop is Linux, way less overhead than docker on mac. I'm simulating request load on a scheduler for compute jobs. In principle those jobs are not generally local, but it helps to stresstest the scheduler.