3 ms·

Treat every line of code you write as both a feature and a liability, then add on a much, much higher cost to test/verify than cost to build. Like say you need

by fny 9y ago

Treat every line of code you write as both a feature and a liability, then add on a much, much higher cost to test/verify than cost to build. Like say you need the feedback of 5 very busy people to get buy in for features and say you also need to have 3 of those very busy people verify the output. Now add in a third-party you need to pay to do security testing. So for any new feature you add, you’re directly adding risk and increasing the cost of formal verification. With these constraints, even if you want to do CD, your CD will end up looking exactly like a waterfall development model. It makes no financial sense otherwise. I work on predictive systems at a hospital, and this ends up being the case for almost every deployment. The costs of making a change and verifying it or adding features is so damn high that you need to think in 3-6 month deployment cycles about everything.

SideburnsOfDoom 9y ago
> say you also need to have 3 of those very busy people verify the output. This assumes that this verification / testing has to be done by a person, and cannot or should not rather be a person producing automation that can be run repeatedly, as much as is possible. Challenging this assumption is one of the points of Continuous Delivery. yes, it's more expensive, slower and more painstaking when there are lives on the line. But this applies regardless of it being automated or not. And automation is more repeatable and scalable than human verification for most of the tests that you want to make.
- fny 9y ago
  We don't use human verification for the bulk of tests, just the last pass. In the healthcare space, you absolutely need to have a few physicians review the final results of a prediction model to make sure they "feel" consistent with prior releases. This even applies for more trivial changes like switching from a bar graph to a line graph. So for both legal and practical reasons, a few physicians will need to give a go ahead. I'm sure you can imagine the legal reasons. A lot of the practical reasons stem from fears that some unknown unknown might trigger a failure that won't be caught by the quality metrics and upstream tests. Anyone in software knows this happens a lot more often than we'd like to admit. Given people will die if the software doesn't work and people will be sued if people die, it would be both irresponsible and risky to depend entirely on an automated test suite for the pre-deployment delivery. Then add a requirement to have a third-party audit the code for HIPAA compliance and security testing, and you can kiss your CD good bye. Even if you want to act like you don't, you still have a milestone, a release date, the evil poison that brings agile thinking to its knees.

by fny 9y ago

SideburnsOfDoom 9y ago

> say you also need to have 3 of those very busy people verify the output. This assumes that this verification / testing has to be done by a person, and cannot or should not rather be a person producing automation that can be run repeatedly, as much as is possible. Challenging this assumption is one of the points of Continuous Delivery. yes, it's more expensive, slower and more painstaking when there are lives on the line. But this applies regardless of it being automated or not. And automation is more repeatable and scalable than human verification for most of the tests that you want to make.

fny 9y ago
We don't use human verification for the bulk of tests, just the last pass. In the healthcare space, you absolutely need to have a few physicians review the final results of a prediction model to make sure they "feel" consistent with prior releases. This even applies for more trivial changes like switching from a bar graph to a line graph. So for both legal and practical reasons, a few physicians will need to give a go ahead. I'm sure you can imagine the legal reasons. A lot of the practical reasons stem from fears that some unknown unknown might trigger a failure that won't be caught by the quality metrics and upstream tests. Anyone in software knows this happens a lot more often than we'd like to admit. Given people will die if the software doesn't work and people will be sued if people die, it would be both irresponsible and risky to depend entirely on an automated test suite for the pre-deployment delivery. Then add a requirement to have a third-party audit the code for HIPAA compliance and security testing, and you can kiss your CD good bye. Even if you want to act like you don't, you still have a milestone, a release date, the evil poison that brings agile thinking to its knees.