4 ms·
How about testing in production? I'm heading in the direction of doing most of my testing there since all to often qa/test/dev environments don't actually repre
by beaker52 8y ago
How about testing in production? I'm heading in the direction of doing most of my testing there since all to often qa/test/dev environments don't actually represent the production system very well and it feels like we're doing it for the sake of it.
Deploy your new service alongside and mirror traffic. Treat your production system as a living organism that you're running tests inside of. Assess the impact/risk of your changes and decide your test strategy. Make large changes in small increments. And most importantly monitor all your changes with metrics, do canary releases when you're rolling out the real deal.
Spend more time testing and validating in production and less time building extensive and usually flaky e2e suites and focus on well pointed smoketests that you run in production.
Reduce the time it takes from writing code to seeing it in production to close to zero and painfully waiting for thousands of unit, integration and e2e tests and getting signoff in n different and yet never quite like production environments suddenly seems like an ancient practice. "How do you know it works?" "The metrics, logs and traces tell me it works"
Of course, it's a living organism, you don't want to screw it up, but there are plenty of ways to isolate your services under test. It definitely isn't something you can just start doing overnight because you'll need tooling and infrastructure to make it happen.
- pytester 8y agoI've had success making an effort to build more realistic and isolated integration tests. One technique in particular that has worked well is to identify small infrastructural problems in my integration tests that prevent me from writing integration tests that replicate real production/QA bugs and, when cost effective, building out infrastructure to enable me to write a test that mimics those conditions. E.g. take the selenium tests that just test with chrome and get them to run on firefox to catch those firefox only bugs. This often allows me to uncover whole classes of bug before the code even leaves the testing environment, never mind hitting prod.