7 ms·

I had never considered this before - but it seems to me that lean development methodologies and CI/CD (and other DevOps practices) will become a defense priorit

I had never considered this before - but it seems to me that lean development methodologies and CI/CD (and other DevOps practices) will become a defense priority, at least a priority of the nations that effectively wage war.

bezospen15 5y ago
Better hope not, you've seen how much bloat it has caused in most organizations. It's trash
ivan_gammel 5y ago
TBH such response does not require CD, only a proper support process where critical fixes can be delivered within hours or minutes. CD is for delivering everything this way, which is not always necessary from business perspective.
- brianwawok 5y ago
  CI/CD says that you can deliver any given bug fix in a few minutes. It doesn’t mean you use it. But IMO most people should.
  - ivan_gammel 5y ago
    CI and CD are different things: you can use CI without CD, e.g. when you have manual process of deployment (if CD is continuous deployment) or long waterfall-like development cycles (if CD is continuous delivery). In highly secure environments CD may not be possible at all, because of formal processes of verification and manual processes of deployment (e.g. I once worked in a project, where deployment process involved copying build artifacts from CI server to an authorized flash drive and papers signed by customer).
    - brianwawok 5y ago
      Fair.
- jcims 5y ago
  This is true but it seems that CD is generally the most efficient method of maintaining this capability for large distributed systems.
  - ivan_gammel 5y ago
    At this point it's probably the right time to ask what do you mean by CD. Continuous Delivery? Continuous Deployment?
- matheweis 5y ago
  The ability to deliver a critical fix on a timeline of minutes to hours is contingent on having well exercised infrastructure and processes in place to do so. If you don’t do CD, it will either be more expensive because you have to maintain two processes or there is a good chance it will fail when you need it most because the critical delivery path wasn’t well exercised.
  - ivan_gammel 5y ago
    > "If you don’t do CD, it will either be more expensive because you have to maintain two processes" Why two? Release process is always the same, only decision to go live is manual and can be based on different criteria in different circumstances. CD is enabled by certain level of automation, which is nice to have, but if you do not aim to have CD, a good manual process can still work and may be actually less expensive in certain scenarios if you consider delivery pipeline TCO.
    - matheweis 5y ago
      For the exact reason you mention: > decision to go live is manual and can be based on different criteria in different circumstances ..this conflicts with previously stated requirement: > a proper support process where critical fixes can be delivered within hours or minutes. That the decision to go live is based on criteria and circumstances implies that there are criteria and circumstances that make a release ineligible to go live at those times. If those criteria and/or circumstances are frequently blocking, then the manual pipeline is not prepared to deliver critical fixes on short notice, because the release pipeline is often blocked (whether CD or not) If those criteria and/or circumstances are infrequently blocking, and the manual release process is also exercised infrequently, then it increases the risk that the manual release process may not work on a critical timeline, because something may be broken but unnoticed from lack of use. If those criteria and/or circumstances are infrequently blocking, and the manual release process is also exercised frequently without problems, then the critical release should be fine. (Although this case seems suspiciously like CD and I’d be a little surprised to hear that the cost of automating such a finely tuned pipeline is significantly greater than the cost of the manpower to run it manually).
thedougd 5y ago
It needs to be, at least for some projects. Unfortunately, it's the organization, internal bureaucracy, and controls that fight it the most. Someone on hacker news said that governance is a glue you pour over an organization to keep it in the current, working form. Changing those mindsets, and avoiding a compliance group from squashing progress, is in my experience nearly impossible.
rocqua 5y ago
I think it extends beyond "do development of software in a lean way", and should also include "make more things configurable with software/firmware instead of hardcoded in hardware". In the starlink case, I imagine they could somehow reconfigure the SDR on the device to get around jamming. If instead the entire chain were done in hardware, then making these kinds of changes would be rather difficult. At the same time, if more things are software more things become targets, so you need to care much more about the actual security of these kinds of update systems.
kcb 5y ago
Lean and Agile have been thrown around in defense contracts for a few years.

bezospen15 5y ago

Better hope not, you've seen how much bloat it has caused in most organizations. It's trash

ivan_gammel 5y ago

TBH such response does not require CD, only a proper support process where critical fixes can be delivered within hours or minutes. CD is for delivering everything this way, which is not always necessary from business perspective.

brianwawok 5y ago
CI/CD says that you can deliver any given bug fix in a few minutes. It doesn’t mean you use it. But IMO most people should.
- ivan_gammel 5y ago
  CI and CD are different things: you can use CI without CD, e.g. when you have manual process of deployment (if CD is continuous deployment) or long waterfall-like development cycles (if CD is continuous delivery). In highly secure environments CD may not be possible at all, because of formal processes of verification and manual processes of deployment (e.g. I once worked in a project, where deployment process involved copying build artifacts from CI server to an authorized flash drive and papers signed by customer).
  - brianwawok 5y ago
    Fair.
jcims 5y ago
This is true but it seems that CD is generally the most efficient method of maintaining this capability for large distributed systems.
- ivan_gammel 5y ago
  At this point it's probably the right time to ask what do you mean by CD. Continuous Delivery? Continuous Deployment?
matheweis 5y ago
The ability to deliver a critical fix on a timeline of minutes to hours is contingent on having well exercised infrastructure and processes in place to do so. If you don’t do CD, it will either be more expensive because you have to maintain two processes or there is a good chance it will fail when you need it most because the critical delivery path wasn’t well exercised.
- ivan_gammel 5y ago
  > "If you don’t do CD, it will either be more expensive because you have to maintain two processes" Why two? Release process is always the same, only decision to go live is manual and can be based on different criteria in different circumstances. CD is enabled by certain level of automation, which is nice to have, but if you do not aim to have CD, a good manual process can still work and may be actually less expensive in certain scenarios if you consider delivery pipeline TCO.
  - matheweis 5y ago
    For the exact reason you mention: > decision to go live is manual and can be based on different criteria in different circumstances ..this conflicts with previously stated requirement: > a proper support process where critical fixes can be delivered within hours or minutes. That the decision to go live is based on criteria and circumstances implies that there are criteria and circumstances that make a release ineligible to go live at those times. If those criteria and/or circumstances are frequently blocking, then the manual pipeline is not prepared to deliver critical fixes on short notice, because the release pipeline is often blocked (whether CD or not) If those criteria and/or circumstances are infrequently blocking, and the manual release process is also exercised infrequently, then it increases the risk that the manual release process may not work on a critical timeline, because something may be broken but unnoticed from lack of use. If those criteria and/or circumstances are infrequently blocking, and the manual release process is also exercised frequently without problems, then the critical release should be fine. (Although this case seems suspiciously like CD and I’d be a little surprised to hear that the cost of automating such a finely tuned pipeline is significantly greater than the cost of the manpower to run it manually).

thedougd 5y ago

It needs to be, at least for some projects. Unfortunately, it's the organization, internal bureaucracy, and controls that fight it the most. Someone on hacker news said that governance is a glue you pour over an organization to keep it in the current, working form. Changing those mindsets, and avoiding a compliance group from squashing progress, is in my experience nearly impossible.

rocqua 5y ago

I think it extends beyond "do development of software in a lean way", and should also include "make more things configurable with software/firmware instead of hardcoded in hardware". In the starlink case, I imagine they could somehow reconfigure the SDR on the device to get around jamming. If instead the entire chain were done in hardware, then making these kinds of changes would be rather difficult. At the same time, if more things are software more things become targets, so you need to care much more about the actual security of these kinds of update systems.

kcb 5y ago

Lean and Agile have been thrown around in defense contracts for a few years.