4 ms·

- Prefer simulators over mocks We're running business critical system for several years now, creating simulators has been one of our "secrets" that contributed

- Prefer simulators over mocks We're running business critical system for several years now, creating simulators has been one of our "secrets" that contributed to the success. Not only our tests are running in as close to production environment as possible, we're also using them for local development where developers can spin up functioning system on own dev machine.

valenterry 5y ago
I agree - but it can be hard to get good ones, e.g. if you need to simulate sql query execution.
- bluGill 5y ago
  Sqlite is a great simulation for SQL. You need to limit your SQL to the subset that us supported by both it and your target database though, which might be a problem.
  - jon-wood 5y ago
    In a world of Docker existing there’s rarely a reason not to just use your target database - I can have a non-production grade Postgres instance up and running in less time than it took me to write this message.
- withinboredom 5y ago
  We spin up a temporary and local MySQL instance per test run with helpers to generate necessary data on-demand. All our tests therefore use real queries on a real db. Due to speed constraints, this db is shared between all the tests you run in that session, so it’s possible to influence tests after yours runs in CI. The reality is that that is pretty easy to detect and it’s caused us to write less brittle code.
  - mirekrusin 5y ago
    We're using mssql, on bootstrap we're running all migrations, seeding database, then backing it up; each test file restores its own database from backup which is much faster than alternatives; e2e tests run against simulated environment that has single database; e2e are split into concurrent runs with distinct set of tests to speed things up.
- mirekrusin 5y ago
  For simulating sql query execution the best is sql server itself.
collyw 5y ago
What do you mean by simulators? An anonymized copy of a database or something similar?
- mirekrusin 5y ago
  It means external services your project depends on in production should: 1. be used directly if possible/practical, if not... 2. ...then it's likely better to write simulators for them as opposed to mocking individual methods on the client in individual tests There are several factors to consider. External services need to be able to run in isolated, temporary environments with short bootstrap time – if that's not possible, it's better to write simulator which provides this functionality. Services must provide determinism, if they don't, simulator should probably be written with control apis to provide this functionality etc. In general the idea can be summarized as opting for using the highest level of functionality available so tests capture as wide code surface (used in production) as possible. It complements, not replaces, lower level methods – ie. it still makes perfect sense to structure code as composition of pure functions with unit tests. It still makes sense to rely on static analysis and do not test in unit tests what is guaranteed by static type system etc. The side effect of simulators means you can bootstrap your project locally in lightweight fashion for development - with all simulated functionality available. For example if you're working on trading application, instead of mocking at the low level prices, order creation calls etc. it's better to write exchange simulator and use it instead – where full lifecycle of an order will work as expected – in tests and locally, when developing application.

valenterry 5y ago

I agree - but it can be hard to get good ones, e.g. if you need to simulate sql query execution.

bluGill 5y ago
Sqlite is a great simulation for SQL. You need to limit your SQL to the subset that us supported by both it and your target database though, which might be a problem.
- jon-wood 5y ago
  In a world of Docker existing there’s rarely a reason not to just use your target database - I can have a non-production grade Postgres instance up and running in less time than it took me to write this message.
withinboredom 5y ago
We spin up a temporary and local MySQL instance per test run with helpers to generate necessary data on-demand. All our tests therefore use real queries on a real db. Due to speed constraints, this db is shared between all the tests you run in that session, so it’s possible to influence tests after yours runs in CI. The reality is that that is pretty easy to detect and it’s caused us to write less brittle code.
- mirekrusin 5y ago
  We're using mssql, on bootstrap we're running all migrations, seeding database, then backing it up; each test file restores its own database from backup which is much faster than alternatives; e2e tests run against simulated environment that has single database; e2e are split into concurrent runs with distinct set of tests to speed things up.
mirekrusin 5y ago
For simulating sql query execution the best is sql server itself.

collyw 5y ago

What do you mean by simulators? An anonymized copy of a database or something similar?

mirekrusin 5y ago
It means external services your project depends on in production should: 1. be used directly if possible/practical, if not... 2. ...then it's likely better to write simulators for them as opposed to mocking individual methods on the client in individual tests There are several factors to consider. External services need to be able to run in isolated, temporary environments with short bootstrap time – if that's not possible, it's better to write simulator which provides this functionality. Services must provide determinism, if they don't, simulator should probably be written with control apis to provide this functionality etc. In general the idea can be summarized as opting for using the highest level of functionality available so tests capture as wide code surface (used in production) as possible. It complements, not replaces, lower level methods – ie. it still makes perfect sense to structure code as composition of pure functions with unit tests. It still makes sense to rely on static analysis and do not test in unit tests what is guaranteed by static type system etc. The side effect of simulators means you can bootstrap your project locally in lightweight fashion for development - with all simulated functionality available. For example if you're working on trading application, instead of mocking at the low level prices, order creation calls etc. it's better to write exchange simulator and use it instead – where full lifecycle of an order will work as expected – in tests and locally, when developing application.