3 ms·

I’m surprised they didn’t mention a superior way to test against reality - record&replay actual responses. Run your tests against actual external services once,

I’m surprised they didn’t mention a superior way to test against reality - record&replay actual responses. Run your tests against actual external services once, and then use those responses in future local-only tests. Periodically refresh your local response log with a cron job.

shardullavekar 3y ago
This very much captures what I am working on. It is (currently java) a sdk dependency + ide plugin, which records input/return values of Java methods. Once recorded you can either instantly replay it on a running process (also works on hot-reloads) OR save it as a fixture which you can replay after a process restart. In our workflow we save these fixtures to git so anyone in the team can replay them. But there are a couple of things I feel are needed on top of this - a flexible assertion framework, currently I am doing a straightforward string equals to identify breaking changes - a mocking feature for downstream calls, currently nothing is mocked out so it is basically an integration test, but as the discussion shows, mocking can be quite handy in a variety of situations
necovek 3y ago
I find simulated fakes that have behaviour that is testable with the same test between a real implementation and fake one to be really superior. It's even better when you've got control of the external dependency (open source project, public API, or internal project) when you can implement something I call "real fakes": use as much as possible from the real code that's fast (i.e. input validation, output adaptation, same basic logic), and have that project export a fake service as a library for easy reuse. This avoids the most common problem with fakes: outdated simulated/mocked implementations. Eg. wouldn't it be wonderful if every project had a "real fake": imagine Postgresql providing really fast fake-pg from the same codebase and tracking their releases, or Amazon SQS providing fake-sqs instead of having to rely on externally provided simulated fakes?

shardullavekar 3y ago

This very much captures what I am working on. It is (currently java) a sdk dependency + ide plugin, which records input/return values of Java methods. Once recorded you can either instantly replay it on a running process (also works on hot-reloads) OR save it as a fixture which you can replay after a process restart. In our workflow we save these fixtures to git so anyone in the team can replay them. But there are a couple of things I feel are needed on top of this - a flexible assertion framework, currently I am doing a straightforward string equals to identify breaking changes - a mocking feature for downstream calls, currently nothing is mocked out so it is basically an integration test, but as the discussion shows, mocking can be quite handy in a variety of situations

necovek 3y ago

I find simulated fakes that have behaviour that is testable with the same test between a real implementation and fake one to be really superior. It's even better when you've got control of the external dependency (open source project, public API, or internal project) when you can implement something I call "real fakes": use as much as possible from the real code that's fast (i.e. input validation, output adaptation, same basic logic), and have that project export a fake service as a library for easy reuse. This avoids the most common problem with fakes: outdated simulated/mocked implementations. Eg. wouldn't it be wonderful if every project had a "real fake": imagine Postgresql providing really fast fake-pg from the same codebase and tracking their releases, or Amazon SQS providing fake-sqs instead of having to rely on externally provided simulated fakes?