5 ms·
I agree with the premise of this post. Instead of mocking, design your components in such a way that they are easy to spin up in tests. Or use containerized ver
by rad_gruchalski 5y ago
I agree with the premise of this post. Instead of mocking, design your components in such a way that they are easy to spin up in tests. Or use containerized versions of services.
I started disliking the idea of mocks a few years ago. I was writing a system based on the Play framework. Play framework used to (still does?) come with a dedicated integration testing environment. The problem with it was that the setup of the testing world slightly differ from the real world. I was bitten a few timer by the real world setup process while the tests were executing flawlessly. In essence, there was no simple way to test the real world construction before deployment to production.
Since then, the only integration tests I accept as real tests are the ones that test the production code path. Database? In containers. Kafka? In containers. etcd? In containers. There are exceptions, though. Proprietary APIs like SQS, or Kinesis, or Firestore are the difficult ones. I usually replace them with hand written memory bound implementations with an abstracted API adhering to some general top level constraints (put / get / consume / publish). This does not prevent errors rooted in the wrong understanding of the design principles of the dependencies, for example, consistency guarantees or delivery ordering, but those are usually possible to cover with additional tests further down the line.
- yepthatsreality 5y agoI personally use mocks or stubs only in unit tests. Everything else should be live test or recorded network responses but always run through live code. Unless the service really depends on it, I do not reccommend setting up local swarm of services. This breaks down if you shift to distributed micro services and has little benefit for a single service app (ex crud + database app, no reason to test the database in the app tests).
- Aeolun 5y agoPersisting data for your test paths is much easier with a database though.
- yepthatsreality 5y agoYes full QA and Prod environments should provide coverage for most cases in which you would not need to setup a database on each test run.
- oriolid 5y agoThe problem is not spinning up services, but setting them up to fail in the way your test case needs them to fail. Testing the happy path isn't that useful, just trying out the system should show that they work. Edge cases and error conditions are what needs testing.
- rad_gruchalski 5y agoThere is no problem with that. A program execution is a result of a given input. Error condition is therefore a result of a set of conditions leading to a fault. There faults can be recreated. The only two things I was not able to test reliably so far are: network latency / failure and disk latency. That is possible with firecracker but I haven’t had a need to go so elaborate yet.