3 ms·
> If you're set up to do A/B tests or deploys with canaries or give potential customers test accounts you're probably able to start testing in production in a s
by derefr 7y ago
> If you're set up to do A/B tests or deploys with canaries or give potential customers test accounts you're probably able to start testing in production in a sane, contained way.
You seem to be assuming 1. some sort of large-horizontal-scale production system with multiple customers, where the impact of a failure can be minimized by minimizing the number of users exposed to new features, and where 2. there's no type of bug in the code that would potentially take down production as a whole.
What if your production system is, say, a bank's ACH reconciliation logic? A medical device? A car? The live server for a popular MMORPG? A telephone backbone switch? A television or radio broadcast station?
In these cases, your software isn't a service with multiple distinct customers that each make requests to it, where you can test your new code on one customer in a thousand; your software is just running and doing something—one, unified something per instance of the system (though that process may track multiple customers)—and if the code is wrong, then the whole system the software operates will fail.
How do you test software for such systems?
Usually by having a "production simulation" whose failure won't kill people or cost a million dollars in lost revenue.
- Roboprog 7y agoThank you for contrasting life building the latest social media platform from what many of the rest of us do. Currently I work on systems to prepare and validate birth and death certificates for the state, counties, hospitals, et al, and this whole “throw it against the wall and see what sticks” methodology doesn’t fly. Nor would it have worked when preparing and presenting investment account information 5 years ago, nor the job 10 years ago processing lawsuit and insurance claim cases and legal bills. Nor any place that I at least have ever worked in the last 30 years.
- TruffleMuffin 7y agoAgree, and I work in the latest 'social media platform' type end. We have many customers. I can assure the author of the post, when those customers pay for enterprise licensing and their system is broken with an obvious bug. The 'we didn't do any testing before hand because staging is a lie' doesn't actually fair well as a valid excuse for anything. In fact, you just look like an unprepared and immature muppet.
- drewcoo 7y agoWell in that case you're not talking about the original scenario where we were talking about whether or not to use a pre-production setup that mimicked production as closely as possible, are you?