3 ms·
I have a test suite with 100s of tests written by someone who was a big fan of this method. When you run the test in isolation, 1s wait here and 2s wait there
by newaccount74 3y ago
I have a test suite with 100s of tests written by someone who was a big fan of this method.
When you run the test in isolation, 1s wait here and 2s wait there is not a big deal. But when you run 200 tests sequentially (because some of them don't work when they run in parallel), then it starts to add up, and your test suite starts feeling sluggish.
- no_wizard 3y agoI'd be asking several questions here: Why wait in the first place and how can I fix the issue of tests not parallelized Both should be red flags in a codebase
- newaccount74 3y agoI mean, there are some valid reasons why the tests are the way they are, and it's not easy to fix them. Re waiting: Test are integration tests that wait for some event. Example: test changes a file on disk, waits for UI to update. We rely on the OS to report file system changes. We don't know how long it takes, because it depends on system load. We could just wait for the next file system event, but we might receive multiple file system events, and since they are processed asynchronously in our app, we don't know how long it takes for the UI to update. We could add some kind of notification when the UI updates to speed up tests, but that would mean adding extra code that might slow down the app just for integration tests. Re parallellizing: Tests are integration tests that run against different databases. Not all databases have perfect transaction support, so sometimes one test interferes with another (eg. one test creates a table, another test fails because that table exists). So parallelising these tests would require multiple copies of each test database, which would cost more money, and spreading tests over multiple test databases would make the test execution code even more complex...