4 ms·

How do you run tests in parallel if part of the logic you are testing is a sql statement? Do you just test them separately? For example, mock out the db when t

How do you run tests in parallel if part of the logic you are testing is a sql statement? Do you just test them separately? For example, mock out the db when testing the app and then sequentially test the db to make sure the sql statement works as expected. However, this explicitly doesn't test the integration.

williamdclt 5y ago
In addition to the database pool approach, you can also write tests so that they are inherently independent. Each test creates and (optionally) deletes its own data, without making assumption about what else is in the database. That's not ideal, as it's hard to know you're not making a hidden assumption.
jdsleppy 5y ago
As one example, Django handles test parallelism by creating N test databases (on the single test database server) and dividing tests into N runners. https://docs.djangoproject.com/en/3.2/ref/django-admin/#envvar-DJANGO_TEST_PROCESSES https://docs.djangoproject.com/en/3.2/ref/django-admin/#envv... You could also have multiple Docker containers running DBs.
- evandwight 5y ago
  Thank you! That's so crazy!
  - williamdclt 5y ago
    Not so crazy, it's very feasible to roll it yourself! Postgres has a "copy database" feature that's very useful (`CREATE DATABASE xxx WITH TEMPLATE yyy`). I saw a project on HN a while ago focused on "managing isolated PostgreSQL databases for your integration tests", never used it but looks like a good idea: https://github.com/allaboutapps/integresql https://github.com/allaboutapps/integresql
bluesnowmonkey 5y ago
One option I’ve used only works if there’s some natural partition of the data like a customer ID. Every test starts by creating a new customer account. Since by design customers can’t see each other’s data, therefore tests can’t interfere with each other and can run in parallel on a single database. After all, in production all your customers are going to be using the database at the same time right? So it needs to work anyway.
- evandwight 5y ago
  Cool. I think I can use this strategy when I add communities.
mtone 5y ago
Here our tests are written in BDD (behavior-driven development) style, mimicking user actions and data expectations. During development, these are run against mocks (either in-memory DB or a mock repository). Individual small scenarios are also combined into realistic long-running processes, for example cases from opened to closed taking various paths. The suite runs in parallel fast and frequently alongside unit tests. Then occasionally like before PR merges, the same scenarios are run against a clone of the production environment to catch any mismatch with the run-time environment, also in parallel (connections) to simulate multi-user usage. Any technical issue prompts an improvement of the mocks and rarely resurfaces. Running these as scripts also doubles as a fake data generator to play with for manual testing, reporting, etc. Last we proceed with some manual testing to validate new changes and pick-up UI-related issues - we don't do UI automation.

williamdclt 5y ago

In addition to the database pool approach, you can also write tests so that they are inherently independent. Each test creates and (optionally) deletes its own data, without making assumption about what else is in the database. That's not ideal, as it's hard to know you're not making a hidden assumption.

jdsleppy 5y ago

As one example, Django handles test parallelism by creating N test databases (on the single test database server) and dividing tests into N runners. https://docs.djangoproject.com/en/3.2/ref/django-admin/#envvar-DJANGO_TEST_PROCESSES https://docs.djangoproject.com/en/3.2/ref/django-admin/#envv... You could also have multiple Docker containers running DBs.

evandwight 5y ago
Thank you! That's so crazy!
- williamdclt 5y ago
  Not so crazy, it's very feasible to roll it yourself! Postgres has a "copy database" feature that's very useful (`CREATE DATABASE xxx WITH TEMPLATE yyy`). I saw a project on HN a while ago focused on "managing isolated PostgreSQL databases for your integration tests", never used it but looks like a good idea: https://github.com/allaboutapps/integresql https://github.com/allaboutapps/integresql

bluesnowmonkey 5y ago

One option I’ve used only works if there’s some natural partition of the data like a customer ID. Every test starts by creating a new customer account. Since by design customers can’t see each other’s data, therefore tests can’t interfere with each other and can run in parallel on a single database. After all, in production all your customers are going to be using the database at the same time right? So it needs to work anyway.

evandwight 5y ago
Cool. I think I can use this strategy when I add communities.

mtone 5y ago

Here our tests are written in BDD (behavior-driven development) style, mimicking user actions and data expectations. During development, these are run against mocks (either in-memory DB or a mock repository). Individual small scenarios are also combined into realistic long-running processes, for example cases from opened to closed taking various paths. The suite runs in parallel fast and frequently alongside unit tests. Then occasionally like before PR merges, the same scenarios are run against a clone of the production environment to catch any mismatch with the run-time environment, also in parallel (connections) to simulate multi-user usage. Any technical issue prompts an improvement of the mocks and rarely resurfaces. Running these as scripts also doubles as a fake data generator to play with for manual testing, reporting, etc. Last we proceed with some manual testing to validate new changes and pick-up UI-related issues - we don't do UI automation.