2 ms·
Something I've learned in Ruby land (prob standard in other places, forgive my ignorance) that seems a bit different than what the article advocates for (fake s
by jrvarela56 3y ago
Something I've learned in Ruby land (prob standard in other places, forgive my ignorance) that seems a bit different than what the article advocates for (fake services):
- Write your service wrapper (eg your logic to interact with Twilio)
- Call the service and record API outputs, save those responses as fixtures that will be returned as response body in your tests without hitting the real thing (eg VCR, WebMock)
- You can now run your tests against old responses (this runs your logic except for getting a real response from the 3rd party; this approach leaves you exposed to API changes and edge cases your logic did not handle)
For the last part, two approaches to overcome this:
- Wrap any new logic in try/catch and report to Sentry: you avoid breaking prod and get info on new edge cases you didn't cover (this may not be feasible if the path where you're inserting new logic into does not work at all without the new feature; address this with thoughtful design/rollout of new features)
- Run new logic side by side to see what happens to the new logic when running in production (https://github.com/github/scientist https://github.com/github/scientist)
I use the first approach bc small startup.
- stephen 3y agoMy caution with recording is that I've found re-recording to get progressively hard/more brittle as time progresses, b/c the test system inevitably drifts further and further from the state it was in when the test was first recorded: https://www.draconianoverlord.com/2018/06/20/skepticism-about-record/replay-tests.html/ https://www.draconianoverlord.com/2018/06/20/skepticism-abou... Granted, it creates more work, but personally I would follow the OP article's advice, and have your production code's tests only run against your in-memory fakes/stubs (which have no coupling to "this is the state of the external system when I happened to make the first recording"). And reserve the record/replays solely for testing your wrapper of the 3rd party API, with the rationale that "just testing the wrapper" will have lower coupling to "what the state of the external system is" vs. your ~10s/100s of production business cases that can be just assume the wrapper primitives like "save this user" / "create this user" work. this