4 ms·

One point missed in this entire calculation of "Continuous Delivery" of systems is the estimated "Cost of a Mistake" [0]. There are many applications where an s

One point missed in this entire calculation of "Continuous Delivery" of systems is the estimated "Cost of a Mistake" [0]. There are many applications where an small error in deployment could lead to a loss of a human life. Code for safety critical systems requires formal checks before being deployed and iterated [1]. There are indeed many many environments where continuous delivery won't work for very good reasons. [0] https://twitter.com/benhamner/status/832785333070606337 https://twitter.com/benhamner/status/832785333070606337 [1] https://en.wikipedia.org/wiki/Static_program_analysis https://en.wikipedia.org/wiki/Static_program_analysis

icebraining 9y ago
Continuous Delivery != Continuous Deployment. The first won't lead to any cost. Also, why can't those formal checks be applied in a CD fashion?
- denzil_correa 9y ago
  The article does not distinguish between the terms delivery and deployment. Some relevant excerpts from the article below > It is often assumed that deploying software more frequently means accepting lower levels of stability and reliability in systems ... > While continuous deployment, the practice of continuously releasing every good build of your software, is mainly limited to cloud- or datacenter-hosted services, continuous delivery—the set of practices described here that enables continuous deployment—can be applied in any domain. ... > In continuous delivery, automated deployments to production-like environments are performed frequently throughout the deployment pipeline,
  - icebraining 9y ago
    The second quote is specifically distinguishing them :) it says that Continuous Delivery enables Continuous Deployment, while making it clear that they don't necessarily apply to the same cases. As for the third quote, notice it says production-like, not production itself. The idea is that you test in an environment that is as close to production as possible. Nobody will die if you deploy a bad build to a real pacemaker - as long as it's not inside an actual person :)
- fny 9y ago
  Treat every line of code you write as both a feature and a liability, then add on a much, much higher cost to test/verify than cost to build. Like say you need the feedback of 5 very busy people to get buy in for features and say you also need to have 3 of those very busy people verify the output. Now add in a third-party you need to pay to do security testing. So for any new feature you add, you’re directly adding risk and increasing the cost of formal verification. With these constraints, even if you want to do CD, your CD will end up looking exactly like a waterfall development model. It makes no financial sense otherwise. I work on predictive systems at a hospital, and this ends up being the case for almost every deployment. The costs of making a change and verifying it or adding features is so damn high that you need to think in 3-6 month deployment cycles about everything.
  - SideburnsOfDoom 9y ago
    > say you also need to have 3 of those very busy people verify the output. This assumes that this verification / testing has to be done by a person, and cannot or should not rather be a person producing automation that can be run repeatedly, as much as is possible. Challenging this assumption is one of the points of Continuous Delivery. yes, it's more expensive, slower and more painstaking when there are lives on the line. But this applies regardless of it being automated or not. And automation is more repeatable and scalable than human verification for most of the tests that you want to make.
    - fny 9y ago
      We don't use human verification for the bulk of tests, just the last pass. In the healthcare space, you absolutely need to have a few physicians review the final results of a prediction model to make sure they "feel" consistent with prior releases. This even applies for more trivial changes like switching from a bar graph to a line graph. So for both legal and practical reasons, a few physicians will need to give a go ahead. I'm sure you can imagine the legal reasons. A lot of the practical reasons stem from fears that some unknown unknown might trigger a failure that won't be caught by the quality metrics and upstream tests. Anyone in software knows this happens a lot more often than we'd like to admit. Given people will die if the software doesn't work and people will be sued if people die, it would be both irresponsible and risky to depend entirely on an automated test suite for the pre-deployment delivery. Then add a requirement to have a third-party audit the code for HIPAA compliance and security testing, and you can kiss your CD good bye. Even if you want to act like you don't, you still have a milestone, a release date, the evil poison that brings agile thinking to its knees.
thinkingkong 9y ago
Sure there are “many” but a tweet and a tangential wikipedia article isnt a figure. There are vastly more businesses where continuous delivery/deployment would be appropriate vs requiring lengthy and exhaustive testing phases.

icebraining 9y ago

Continuous Delivery != Continuous Deployment. The first won't lead to any cost. Also, why can't those formal checks be applied in a CD fashion?

denzil_correa 9y ago
The article does not distinguish between the terms delivery and deployment. Some relevant excerpts from the article below > It is often assumed that deploying software more frequently means accepting lower levels of stability and reliability in systems ... > While continuous deployment, the practice of continuously releasing every good build of your software, is mainly limited to cloud- or datacenter-hosted services, continuous delivery—the set of practices described here that enables continuous deployment—can be applied in any domain. ... > In continuous delivery, automated deployments to production-like environments are performed frequently throughout the deployment pipeline,
- icebraining 9y ago
  The second quote is specifically distinguishing them :) it says that Continuous Delivery enables Continuous Deployment, while making it clear that they don't necessarily apply to the same cases. As for the third quote, notice it says production-like, not production itself. The idea is that you test in an environment that is as close to production as possible. Nobody will die if you deploy a bad build to a real pacemaker - as long as it's not inside an actual person :)
fny 9y ago
Treat every line of code you write as both a feature and a liability, then add on a much, much higher cost to test/verify than cost to build. Like say you need the feedback of 5 very busy people to get buy in for features and say you also need to have 3 of those very busy people verify the output. Now add in a third-party you need to pay to do security testing. So for any new feature you add, you’re directly adding risk and increasing the cost of formal verification. With these constraints, even if you want to do CD, your CD will end up looking exactly like a waterfall development model. It makes no financial sense otherwise. I work on predictive systems at a hospital, and this ends up being the case for almost every deployment. The costs of making a change and verifying it or adding features is so damn high that you need to think in 3-6 month deployment cycles about everything.
- SideburnsOfDoom 9y ago
  > say you also need to have 3 of those very busy people verify the output. This assumes that this verification / testing has to be done by a person, and cannot or should not rather be a person producing automation that can be run repeatedly, as much as is possible. Challenging this assumption is one of the points of Continuous Delivery. yes, it's more expensive, slower and more painstaking when there are lives on the line. But this applies regardless of it being automated or not. And automation is more repeatable and scalable than human verification for most of the tests that you want to make.
  - fny 9y ago
    We don't use human verification for the bulk of tests, just the last pass. In the healthcare space, you absolutely need to have a few physicians review the final results of a prediction model to make sure they "feel" consistent with prior releases. This even applies for more trivial changes like switching from a bar graph to a line graph. So for both legal and practical reasons, a few physicians will need to give a go ahead. I'm sure you can imagine the legal reasons. A lot of the practical reasons stem from fears that some unknown unknown might trigger a failure that won't be caught by the quality metrics and upstream tests. Anyone in software knows this happens a lot more often than we'd like to admit. Given people will die if the software doesn't work and people will be sued if people die, it would be both irresponsible and risky to depend entirely on an automated test suite for the pre-deployment delivery. Then add a requirement to have a third-party audit the code for HIPAA compliance and security testing, and you can kiss your CD good bye. Even if you want to act like you don't, you still have a milestone, a release date, the evil poison that brings agile thinking to its knees.

thinkingkong 9y ago

Sure there are “many” but a tweet and a tangential wikipedia article isnt a figure. There are vastly more businesses where continuous delivery/deployment would be appropriate vs requiring lengthy and exhaustive testing phases.