3 ms·
Some places you simply can't. Medical devices, airplanes, etc. But non-critical services? Sure
by tomrod 5y ago
Some places you simply can't. Medical devices, airplanes, etc.
But non-critical services? Sure
- jdauriemma 5y agoYou absolutely can and should test in prod in these cases.
- frozenport 5y agoWe call that FDA testing
- ekianjo 5y agoFDA does not test anything.
- Supermancho 5y agoI'm not sure how this is a useful observation. Units don't test anything, but it's still called Unit Testing.
- thewakalix 5y agoThey test my patience!
- anaganisk 5y agoThey will and do something called acceptance testing, which is nothing but testing in prod.
- detaro 5y agoNot quite. Nobody acceptance tests an airliner on a passenger flight. If we acceptance test a medical diagnostics device, we don't run the test the doctor later relies on with the DUT.
- jdauriemma 5y agoThis is a common misconception about testing in prod. It’s not about the logic. To quote the article: “Once you deploy, you aren’t testing code anymore, you’re testing systems.”
- detaro 5y agoI don't quite get the objection. For me, "testing in prod" is "we observe the actual running system, with production traffic and users directly interacting with it and its results being live". That's not quite what "acceptance testing" in the mentioned-above domains is. If you have an acceptance testing stage, prod follows it.
- joshuamorton 5y agoI think the argument breaks down when you have systems as opposed to objects. You can't acceptance test "Google's search", because at a minimum doing so would require some kind of reverse proxy that is itself part of the system and can't be acceptance tested without...and turtles etc. Another way of putting this might be that there isn't a good way to acceptance test "airplane manufacturing companies". There isn't a set of acceptance tests you can run to ensure that Boeing is performing to spec before having it build real airplanes.
- detaro 5y agoMaybe, I don't feel particularly strongly either way about if it can be applied to the kind of large services you are thinking of. But also wasn't really the point of my comment, I merely rejected the claim that acceptance testing in fields where it is done is the same as testing in prod.
- VBprogrammer 5y agoPerhaps the closest analogy would be deploying to a limited subset of users on closely monitored boxes. It's clearly not perfect though, except perhaps if the test aircraft turned itself into a crater on the runway, there is very little one aircraft can do to effect the function of all others. In software it's hard to get this kind of guarantee, a new delete with a missing where clause in your canary environment is going to take some time to clean up after (assuming you have backups etc).