3 ms·
The issue with testing environments is that they almost always differ from the production environment in important ways. Cloud and infrastructure-as-code can mi
by tormeh 5y ago
The issue with testing environments is that they almost always differ from the production environment in important ways. Cloud and infrastructure-as-code can mitigate this to a certain extent, but I doubt this issue can ever be fully eliminated. Good CI pipelines should always be our first line of defense. Testing environments sometimes sound more appealing because you don't have to write tests, but that's a trap.
- cestith 5y agoOne important way testing environments and production environments necessarily differ is very germane to discussion of this problem. If you have an issue in 1 of 64 cores on a CPU on 1 out of 1000 CPUs, what are the chances your testing environment will ever see the issue? Do you have 10,000 CPUs in your testing environment? I know I don't. At scale we really do need to think about monitoring, automated diagnosis, and recovery.
- hinkley 5y agoBefore cloud it was frustratingly common for QA to have lesser hardware than the production hardware, but the problem with that is that at best you could get a vendor to overnight you replacements for a hardware failure. Meanwhile if your QA team has the same hardware as prod, you can cannibalize a QA machine to replace the production hardware that failed. Your QA probably cares less about being down overnight than your customers do.
- pm90 5y agoI think we need to kill the fantasy of having testing environments for most software. For stuff like Airplane control, pacemakers etc. it’s absolutely worth it. For most organizations serving APIs with an acceptable nonzero failure rate, using canary deployments and other techniques provide almost all the value you would need.