14 ms·

Department of Defense: Software Is Never Done (2019) [pdf]

168 points by jasonhansel 4y ago

mLuby 4y ago
> In a year replete with reports on artificial intelligence, quantum computing, and blockchain, may seem mundane to write about software, but software is the foundation of all things digital, and the Department does it exceedingly poorly. I take issue with the title. Software is constantly "done" and then new requirements arrive, causing a new version of the completed software to be built. A suspension bridge is considered "done" even if it's getting continuously repainted.
- gilbetron 4y ago
  You are thinking in terms of software in the non-government world. Contractors try really hard never to finish software so they can keep extending their contracts. This document is about the military world, and if you don't have experience in it, it can be hard to interpret!
  - sylware 4y ago
    military world only? Look at how the TV series industry is hard trying with cliffhangers or story telling arcs without a closure... This issue runs much deeper: It is how the economy operates: you need a roof to protect you, every day, you need to eat, every day, until you die in decades. Many economic activities, if done honestly and fairly, cannot justify a permanent income.
- gardenhedge 4y ago
  Nearly all software has dependencies that need to be updated over time. How does that work against the bride analogy?
cushychicken 4y ago
I didn't read through the whole thing, but I did skip to the section of the ten recommendations and each one makes a whole heap of sense to me. Anyone know if Congress has done any of these things in the last three years, since this report has come out?
gilbetron 4y ago
I was a DoD contractor (the coding type, not the shooting type) for almost 15 years, and this is a really interesting read. I've heard some software devs state that "the waterfall method was never really used". Those devs were not DoD software contractors for almost 15 years. DoD was and is all about the waterfall model. I have spreadsheet PTSD from seeing so many waterfall diagrams and project manager's Gantt charts. I was working on a peanut-size project for the Navy (around $5-10million over 5 years) when Agile began picking up steam (mid 00s). I introduced the team to it and we began doing Agile to decent effect (at least in comparison to waterfall), and introduced it to our Navy sponsors. I remember some higher-ranking individual ask, "wait, so this method can actually deliver software?" Or something like that. Of course, Agile is no panacea, but the iterative nature was waaaay better than what the DoD was doing. I really like how they talk about "individuals of talent" in the document. Much of the DoD focuses around process, to great effect. The problem is that you can't take a process and put software drones on it and deliver software of any complexity. That goes against much (but not all!) of the DoD paradigm, it is interesting to see they might be changing. Of course, they will meet great resistance, after all, the first job of a successful contractor is to extract as much money as possible from the government with as little work as possible. "Never finish a contract" was the mantra we had learned from the giant contractors. Glad to be out of there, but it was a really interesting experience!
- jdeaton 4y ago
  Just out of curiosity, how does someone with in-demand, highly-transferable skills like software development find themselves working in a place they generally dislike for 15 years without leaving sooner?
  - gilbetron 4y ago
    Oh, I had an awesome time for much of it. Small business with many great friends and we got into many antics over those years through thick and thin. Also, I got to do really cool, nearly constant greenfield work, like flight simulators, robot simulators, radar simulations, robotics, haptics, and then there was some non-DoD work like a virtual gorilla VR experience, gaming stuff, and all kinds of funs times and trips to Game Dev and E3 conferences. I once got to fly on a mid-air refueling mission. Got to see one of the Mars rovers as it was being built. Went on a Naval frigate. Ran a class for some pilot trainees in a building next to the Blue Angels home base. But there was lots of boring bullshit in there too ;)
scythmic_waves 4y ago
The 10 recommendations found on the "cheat sheet" (page xv): Line of Effort A (Congress and OSD): Refactor statutes, regulations, and processes for software * [A1] Establish one or more new acquisition pathways for software that prioritize continuous integration and delivery of working software in a secure manner, with continuous oversight from automated analytics * [A2] Create a new appropriation category for software capability delivery that allows (relevant types of) software to be funded as a single budget item, with no separation between RDT&E, production, and sustainment Line of Effort B (OSD and Services): Create and maintain cross-program/cross-Service digital infrastructure * [B1] Establish and maintain digital infrastructure within each Service or Agency that enables rapid deployment of secure software to the field, and incentivize its use by contractors * [B2] Create, implement, support, and use fully automatable approaches to testing and evaluation (T&E), including security, that allow high-confidence distribution of software to the field on an iterative basis * [B3] Create a mechanism for Authorization to Operate (ATO) reciprocity within and between programs, Services, and other DoD agencies to enable sharing of software platforms, components, and infrastructure and rapid integration of capabilities across (hardware) platforms, (weapon) systems, and Services Line of Effort C (Services and OSD): Create new paths for digital talent (especially internal talent) * [C1] Create software development units in each Service consisting of military and civilian personnel who develop and deploy software to the field using DevSecOps practices * [C2] Expand the use of (specialized) training programs for CIOs, SAEs, PEOs, and PMs that provide (hands-on) insight into modern software development (e.g., Agile, DevOps, DevSecOps) and the authorities available to enable rapid acquisition of software Line of Effort D (DoD and industry): Change the practice of how software is procured and developed * [D1] Require access to source code, software frameworks, and development toolchains —- with appropriate IP rights —- for DoD-specific code, enabling full security testing and rebuilding of binaries from source * [D2] Make security a first-order consideration for all software-intensive systems, recognizing that security-at-the-perimeter is not enough * [D3] Shift from the use of rigid lists of requirements for software programs to a list of desired features and required interfaces/characteristics to avoid requirements creep, overly ambitious requirements, and program delays
dathinab 4y ago
I know it's not what they mean and I agree with the general idea. But for very small very well contained libraries I would argue that they sometime _are_ done (for many years, not forever; in a sense that any changes you can come up with add no relevant value at all, including no relevant perf. improvements or better maintainability). At least if written in a language with nice forward and backward compatibility, cross platform support and a stable "base" ecosystem. (So not C,C++ due to not default package manager/build system and too much divergence between platforms (e.g. unsigned long), not JS because their core build tooling changes to much (through it's getting better!), etc.)
- legrande 4y ago
  > But for very small very well contained libraries I would argue that they sometime _are_ done https://en.wikipedia.org/wiki/Unix_philosophy#Do_One_Thing_and_Do_It_Well https://en.wikipedia.org/wiki/Unix_philosophy#Do_One_Thing_a... > As stated by McIlroy, and generally accepted throughout the Unix community, Unix programs have always been expected to follow the concept of DOTADIW, or "Do One Thing And Do It Well." There are limited sources for the acronym DOTADIW on the Internet, but it is discussed at length during the development and packaging of new operating systems, especially in the Linux community.
jasonhansel 4y ago
A question I have: do people working on DoD software ever get to use it themselves? I assume many of them don't (if they're working on, say, a weapons system). This situation often (IMHO) makes it much harder to deliver high-quality software. No amount of surface-level UI testing replaces the knowledge you get from actually using a system yourself.
lumb63 4y ago
I want to read more of this later. After 4 years in DoD contracting, at a quick glance, some of this hits the nail right on the head: >Software is made by people and for people, so digital talent matters. The company I was at did not acknowledge this. To them, engineering hours were fungible. Two weeks from an engineer who is doing the bare minimum and coasting to retirement (in DoD contracting, there are lots of these type) is the same as a mid-level engineer who is up-to-date on the latest technology and working their hardest to push the project forward. The latter is usually paid less than half the former in the industry. >Software is different than hardware (and not all software is the same). What works in other industries doesn't always work in defense. It's a unique space. CI? Good luck running automated "intercept a hypersonic missile" tests. This doesn't get enough attention. However, there is a lot that can be done in pure software testing, even for DoD applications. They view it too much like hardware development and that hamstrings their software efforts. The bigger issue that is industry-wide that I noticed in my time there, is that the incentive system in the industry is broken. Contractors take an educated guess up front on what it will cost them to build a system for the government, and the government chooses from those bids, weighing capabilities, cost schedule, etc. The contractor who wins, then tracks their progress to completion by how many hours they have worked on the contract, relative to their total. This is how they get paid. For instance, if it is predicted to take 100 hours to deliver a system, and that system is $1 million, each hour would earn the contractor something like $10,000 (obviously there are other non-labor costs and this is a hand waving simplification, but to an extent is true). So, there is limited (bonuses that are tiny relative to the value of the contract) incentive to come in on time. There is a great incentive to over-hire, pay staff to sit around doing nothing, and collect money for it. This disincentivizes innovation, because you don't want to have a predicted cost that is astonishingly low for your next contract; you want it to be just low enough to undercut your next competitor. That's how you maximize profits, and us taxpayers get a terrible deal.
- giaour 4y ago
  There are multiple contract types, and you're correct that the "cost-plus" paradigm (where the contract covers the contractor's costs plus a profit margin) has some perverse incentives. I saw a number of smaller efforts at Medicare use the "firm fixed price" paradigm (where the total cost of the contract is agreed to up front), which allows the contractor to keep whatever they save by delivering early or using a smaller than predicted staff but makes the contractor liable for any cost or staffing overages. That system seemed to produce better aligned incentives, but it's more difficult to apply to larger projects.
jongjong 4y ago
As a software developer with 15 years of experience who produces completed software products, I'm wary of this mindset because it's often used as an excuse to tolerate or reward incompetence. Software can be considered done with respect to any given set of requirements. Now if the requirements change, then of course, the software may need to be updated - but even then, it should be designed in such a way that the impact of requirement changes are minimal. Software should, to some extent, anticipate requirement changes. Of course this is not easy, but to claim that it's impossible is to admit defeat. You need highly skilled people making those architectural decisions. Some software is subjected to constantly changing requirements and these can be said to never be done but that's far from being the case for all software. Libraries and modules are software too and these should be generic enough to handle requirement changes without code changes since they should not concern themselves with the business domain. I have completed open source projects which did not need to be updated in years and worked just as well. I've worked on other projects which had to be updated because of vulnerabilities found in dependencies but these did not require any code changes on my end. I consider it a huge red flag if some library was working fine for 5 years and then people decided to change its interface in a non-trivial way. I was tolerant when async/await was rolled out to major programming languages; this affected and continues to affect dependencies but it's a meaningful conceptual improvement and helps with future maintainability. I hope that programming standards organizations will slow down after that in terms of changes which affect function interfaces because these decisions have huge flow-on impacts. It's totally possible to produce completed software (especially libraries and frameworks) but the industry has to aim for it and expect it. The current goal of industry seems to be job creation, not the creation of robust working products. Accepting that software can never be done right from the beginning is going to be a disaster. It should only be tolerated in the context of requirement changes.
- jasonhansel 4y ago
  As a fellow software developer, I respectfully disagree. While a piece of software may satisfy its requirements, it will never satisfy them perfectly or optimally. All software contains bugs, unexpected failure modes, and undiscovered security vulnerabilities. I am convinced that there are essentially zero exceptions to these claims. Furthermore, the environment in which software runs must change over time. This causes "bit rot" as (say) dependencies stop receiving security updates or platforms become unsupported. Finally, there's the human factor. When software runs on old technologies, it will become progressively harder for new developers to understand. There will consequently often be a steady decline in available talent. For these reasons, even if the requirements of a piece of software never change, it will always require ongoing maintenance to keep it running, much like a piece of physical infrastructure. (edit: typo)
deleted 4y ago
[deleted]
wrp 4y ago
The attitude toward the development process expressed here is not uniformly accepted in all programming subcultures. There is a major cultural divide between embedded and server/desktop programming. My example is very old, but I had occasion to discuss software development with embedded systems engineers from Air Force projects in the pre-Ada days. They were all vehemently against the idea of software maintenance. They felt that their job was to deliver a "correct" implementation of a fully specified system.
- smadge 4y ago
  That model makes sense for aircraft, in my opinion. The software isn’t a standalone product that can be independently continuously improved. It’s tightly integrated with safety critical physical systems. You need to do the upfront work of specifying how the software and hardware interact and do the complex systems safety analysis. Any change to the software specification after design can have unintended effects on the overall behavior of the aircraft.
neilv 4y ago
> As Steve Jobs observed, 8 one of the major differences between hardware and software is that for hardware the “dynamic range” (ratio between the best in class and average performance) is, at most, 2:1. But, the difference between the best software developer and an average software developer can be 50:1, or even 100:1, and putting great developers on a team with other great developers amplifies this effect. Today, in DoD and the industrial base that supports it, the people with the necessary skills exist, but instead of taking advantage of their skills we put them in environments where it is difficult for them to be effective. DoD does not take advantage of already existing military and civilian personnel expertise by offering pay bonuses, career paths that provide the ability to stay in their specialization, or access to early promotions. Skilled software engineers and the related specialties that are part of the overall software ecosystem need to be treated as a special force; the United States must harness their talent for the great benefits that it can provide. As if we software people didn't have overinflated egos already... This DoD report talks about 100x developers existing (and of course HNers imagine ourselves towards that better-than-average end of the scale). The DoD here even uses the (lowercase) term "special force", which sounds like we're not nerd typists, but super-cool elite operators.
- 4648283 4y ago
  Ironically I suspect that wind l while the idea will feed many egos the reality would crush or frustrate those same people. If the DoD certifies someone as a 100x developer it would be after crafting a ruthlessly banal evaluation system that would likely lack the romantic attitude towards software many developers have in my experience. Especially if the criteria included some social skills component that determined part of a developers rating by how well they synergize with similarly rated peers.
  - neilv 4y ago
    I think the DoD has had many outstanding R&D successes, hiring or contracting elite engineers and scientists to do amazing things, which beat most of what we see from tech private sector B2C and B2B businesses. That's one place you'll need some people who can find brilliant solutions or insights that most people simply cannot, no matter whether you throw 100x times the number of people at the problem. But if they're having trouble attracting and retaining as many 1x-3x everyday software developers as they'd like, I agree, it'd be easy to imagine an attempt that fails miserably due to stereotypical bureaucracy. Paying close to FAANG TC levels would be a start, since it would cover the monetary factor, and i think the prestige factor would quickly follow. (I bet few people think adtech, fintech, or yet-another-microblogging/messaging/profile-app are actually cool anymore -- only the money is what makes those companies prestigious.) Maybe pay 80% of FAANG, so the FAANGs will still get the people who only want money, and you offer better non-monetary value to make up that 20%. (80% should be close enough they can afford to live in the same neighborhoods and send their kids to the same schools as FAANG, rather than be lower class that could upgrade just by grinding Leetcode for a few months.) Then manage those units like they are very serious excellent product development organizations where everyone wants to work. And figure out how to do the culture better than all the companies that got too much into class thinking, and too much jumping through the hoops for interview and pursuit of promotions, rather than focus on the mission and team.
killjoywashere 4y ago
The problem isn't that DoD requires waterfall for software. The problem is the Planning Programming Budgeting Execution cycle. If you solve a problem, you have at least an 18 month hole in the schedule. This drives PMs to waterfall so the can project funding out over the POM hole. It's profoundly broken and kept that way by Congress critters in the pocket of the prime integrators.
ngcc_hk 4y ago
What is meant by done? As part of system it co-evolves with it and due with it. It dies with the system it sort of “parasite” on or it is the key part of the system or it is the system. Still evolve until death.
einpoklum 4y ago
It's like reading a report by Darth Vader about the "national defense" of the galactic empire, explaining what's necessary for crushing "our adversaries" the rebels. Chill down my spine.
mihaigalos 4y ago
I feel Scrum has evolved into something contradicting the original Agile practices. These were more of a template on how to develop, not a concrete recipe. Scrum then established a process where there was need for a family of processes, continuously evolving, ever changing. The very fact that "you absolutely have to have a specific ceremony (i.e.: Retro when no topics)" is a red flag which imho puts the cart before the horse. Why is there a "Scrum Master" anyway, if individuals are encouraged to interact and own the process? Suppose an individual wants to change the process, and reaches consensus within the team/community. Why would a Scrum Master / Manager even be able to veto any decision? Why is there a Sprint when SW development is actually a marathon? Uncle Bob talks about this in several of his Clean Methodology books. I don't think that Agile is dead [1], although it has been taken over by people eager to sell you a *cough* certification (Scrum or any other). It's not dead because its genius is that it's constantly evolving, and you can always interpret it differently since it's only a template. Scrum, on the other hand.. well.. For the future, I look forward to not putting an equal sign between Scrum and Agile, even though apparently Scrum is how you "do" Agile, according to the industry. [1] https://www.simplethread.com/agile-at-20-the-failed-rebellion/ https://www.simplethread.com/agile-at-20-the-failed-rebellio...

168 points by jasonhansel 4y ago

> In a year replete with reports on artificial intelligence, quantum computing, and blockchain, may seem mundane to write about software, but software is the foundation of all things digital, and the Department does it exceedingly poorly. I take issue with the title. Software is constantly "done" and then new requirements arrive, causing a new version of the completed software to be built. A suspension bridge is considered "done" even if it's getting continuously repainted.

gilbetron 4y ago
You are thinking in terms of software in the non-government world. Contractors try really hard never to finish software so they can keep extending their contracts. This document is about the military world, and if you don't have experience in it, it can be hard to interpret!
- sylware 4y ago
  military world only? Look at how the TV series industry is hard trying with cliffhangers or story telling arcs without a closure... This issue runs much deeper: It is how the economy operates: you need a roof to protect you, every day, you need to eat, every day, until you die in decades. Many economic activities, if done honestly and fairly, cannot justify a permanent income.
gardenhedge 4y ago
Nearly all software has dependencies that need to be updated over time. How does that work against the bride analogy?

cushychicken 4y ago

I didn't read through the whole thing, but I did skip to the section of the ten recommendations and each one makes a whole heap of sense to me. Anyone know if Congress has done any of these things in the last three years, since this report has come out?

gilbetron 4y ago

I was a DoD contractor (the coding type, not the shooting type) for almost 15 years, and this is a really interesting read. I've heard some software devs state that "the waterfall method was never really used". Those devs were not DoD software contractors for almost 15 years. DoD was and is all about the waterfall model. I have spreadsheet PTSD from seeing so many waterfall diagrams and project manager's Gantt charts. I was working on a peanut-size project for the Navy (around $5-10million over 5 years) when Agile began picking up steam (mid 00s). I introduced the team to it and we began doing Agile to decent effect (at least in comparison to waterfall), and introduced it to our Navy sponsors. I remember some higher-ranking individual ask, "wait, so this method can actually deliver software?" Or something like that. Of course, Agile is no panacea, but the iterative nature was waaaay better than what the DoD was doing. I really like how they talk about "individuals of talent" in the document. Much of the DoD focuses around process, to great effect. The problem is that you can't take a process and put software drones on it and deliver software of any complexity. That goes against much (but not all!) of the DoD paradigm, it is interesting to see they might be changing. Of course, they will meet great resistance, after all, the first job of a successful contractor is to extract as much money as possible from the government with as little work as possible. "Never finish a contract" was the mantra we had learned from the giant contractors. Glad to be out of there, but it was a really interesting experience!

jdeaton 4y ago
Just out of curiosity, how does someone with in-demand, highly-transferable skills like software development find themselves working in a place they generally dislike for 15 years without leaving sooner?
- gilbetron 4y ago
  Oh, I had an awesome time for much of it. Small business with many great friends and we got into many antics over those years through thick and thin. Also, I got to do really cool, nearly constant greenfield work, like flight simulators, robot simulators, radar simulations, robotics, haptics, and then there was some non-DoD work like a virtual gorilla VR experience, gaming stuff, and all kinds of funs times and trips to Game Dev and E3 conferences. I once got to fly on a mid-air refueling mission. Got to see one of the Mars rovers as it was being built. Went on a Naval frigate. Ran a class for some pilot trainees in a building next to the Blue Angels home base. But there was lots of boring bullshit in there too ;)

scythmic_waves 4y ago

The 10 recommendations found on the "cheat sheet" (page xv): Line of Effort A (Congress and OSD): Refactor statutes, regulations, and processes for software * [A1] Establish one or more new acquisition pathways for software that prioritize continuous integration and delivery of working software in a secure manner, with continuous oversight from automated analytics * [A2] Create a new appropriation category for software capability delivery that allows (relevant types of) software to be funded as a single budget item, with no separation between RDT&E, production, and sustainment Line of Effort B (OSD and Services): Create and maintain cross-program/cross-Service digital infrastructure * [B1] Establish and maintain digital infrastructure within each Service or Agency that enables rapid deployment of secure software to the field, and incentivize its use by contractors * [B2] Create, implement, support, and use fully automatable approaches to testing and evaluation (T&E), including security, that allow high-confidence distribution of software to the field on an iterative basis * [B3] Create a mechanism for Authorization to Operate (ATO) reciprocity within and between programs, Services, and other DoD agencies to enable sharing of software platforms, components, and infrastructure and rapid integration of capabilities across (hardware) platforms, (weapon) systems, and Services Line of Effort C (Services and OSD): Create new paths for digital talent (especially internal talent) * [C1] Create software development units in each Service consisting of military and civilian personnel who develop and deploy software to the field using DevSecOps practices * [C2] Expand the use of (specialized) training programs for CIOs, SAEs, PEOs, and PMs that provide (hands-on) insight into modern software development (e.g., Agile, DevOps, DevSecOps) and the authorities available to enable rapid acquisition of software Line of Effort D (DoD and industry): Change the practice of how software is procured and developed * [D1] Require access to source code, software frameworks, and development toolchains —- with appropriate IP rights —- for DoD-specific code, enabling full security testing and rebuilding of binaries from source * [D2] Make security a first-order consideration for all software-intensive systems, recognizing that security-at-the-perimeter is not enough * [D3] Shift from the use of rigid lists of requirements for software programs to a list of desired features and required interfaces/characteristics to avoid requirements creep, overly ambitious requirements, and program delays

dathinab 4y ago

I know it's not what they mean and I agree with the general idea. But for very small very well contained libraries I would argue that they sometime _are_ done (for many years, not forever; in a sense that any changes you can come up with add no relevant value at all, including no relevant perf. improvements or better maintainability). At least if written in a language with nice forward and backward compatibility, cross platform support and a stable "base" ecosystem. (So not C,C++ due to not default package manager/build system and too much divergence between platforms (e.g. unsigned long), not JS because their core build tooling changes to much (through it's getting better!), etc.)

legrande 4y ago
> But for very small very well contained libraries I would argue that they sometime _are_ done https://en.wikipedia.org/wiki/Unix_philosophy#Do_One_Thing_and_Do_It_Well https://en.wikipedia.org/wiki/Unix_philosophy#Do_One_Thing_a... > As stated by McIlroy, and generally accepted throughout the Unix community, Unix programs have always been expected to follow the concept of DOTADIW, or "Do One Thing And Do It Well." There are limited sources for the acronym DOTADIW on the Internet, but it is discussed at length during the development and packaging of new operating systems, especially in the Linux community.

jasonhansel 4y ago

A question I have: do people working on DoD software ever get to use it themselves? I assume many of them don't (if they're working on, say, a weapons system). This situation often (IMHO) makes it much harder to deliver high-quality software. No amount of surface-level UI testing replaces the knowledge you get from actually using a system yourself.

lumb63 4y ago

I want to read more of this later. After 4 years in DoD contracting, at a quick glance, some of this hits the nail right on the head: >Software is made by people and for people, so digital talent matters. The company I was at did not acknowledge this. To them, engineering hours were fungible. Two weeks from an engineer who is doing the bare minimum and coasting to retirement (in DoD contracting, there are lots of these type) is the same as a mid-level engineer who is up-to-date on the latest technology and working their hardest to push the project forward. The latter is usually paid less than half the former in the industry. >Software is different than hardware (and not all software is the same). What works in other industries doesn't always work in defense. It's a unique space. CI? Good luck running automated "intercept a hypersonic missile" tests. This doesn't get enough attention. However, there is a lot that can be done in pure software testing, even for DoD applications. They view it too much like hardware development and that hamstrings their software efforts. The bigger issue that is industry-wide that I noticed in my time there, is that the incentive system in the industry is broken. Contractors take an educated guess up front on what it will cost them to build a system for the government, and the government chooses from those bids, weighing capabilities, cost schedule, etc. The contractor who wins, then tracks their progress to completion by how many hours they have worked on the contract, relative to their total. This is how they get paid. For instance, if it is predicted to take 100 hours to deliver a system, and that system is $1 million, each hour would earn the contractor something like $10,000 (obviously there are other non-labor costs and this is a hand waving simplification, but to an extent is true). So, there is limited (bonuses that are tiny relative to the value of the contract) incentive to come in on time. There is a great incentive to over-hire, pay staff to sit around doing nothing, and collect money for it. This disincentivizes innovation, because you don't want to have a predicted cost that is astonishingly low for your next contract; you want it to be just low enough to undercut your next competitor. That's how you maximize profits, and us taxpayers get a terrible deal.

giaour 4y ago
There are multiple contract types, and you're correct that the "cost-plus" paradigm (where the contract covers the contractor's costs plus a profit margin) has some perverse incentives. I saw a number of smaller efforts at Medicare use the "firm fixed price" paradigm (where the total cost of the contract is agreed to up front), which allows the contractor to keep whatever they save by delivering early or using a smaller than predicted staff but makes the contractor liable for any cost or staffing overages. That system seemed to produce better aligned incentives, but it's more difficult to apply to larger projects.

jongjong 4y ago

As a software developer with 15 years of experience who produces completed software products, I'm wary of this mindset because it's often used as an excuse to tolerate or reward incompetence. Software can be considered done with respect to any given set of requirements. Now if the requirements change, then of course, the software may need to be updated - but even then, it should be designed in such a way that the impact of requirement changes are minimal. Software should, to some extent, anticipate requirement changes. Of course this is not easy, but to claim that it's impossible is to admit defeat. You need highly skilled people making those architectural decisions. Some software is subjected to constantly changing requirements and these can be said to never be done but that's far from being the case for all software. Libraries and modules are software too and these should be generic enough to handle requirement changes without code changes since they should not concern themselves with the business domain. I have completed open source projects which did not need to be updated in years and worked just as well. I've worked on other projects which had to be updated because of vulnerabilities found in dependencies but these did not require any code changes on my end. I consider it a huge red flag if some library was working fine for 5 years and then people decided to change its interface in a non-trivial way. I was tolerant when async/await was rolled out to major programming languages; this affected and continues to affect dependencies but it's a meaningful conceptual improvement and helps with future maintainability. I hope that programming standards organizations will slow down after that in terms of changes which affect function interfaces because these decisions have huge flow-on impacts. It's totally possible to produce completed software (especially libraries and frameworks) but the industry has to aim for it and expect it. The current goal of industry seems to be job creation, not the creation of robust working products. Accepting that software can never be done right from the beginning is going to be a disaster. It should only be tolerated in the context of requirement changes.

jasonhansel 4y ago
As a fellow software developer, I respectfully disagree. While a piece of software may satisfy its requirements, it will never satisfy them perfectly or optimally. All software contains bugs, unexpected failure modes, and undiscovered security vulnerabilities. I am convinced that there are essentially zero exceptions to these claims. Furthermore, the environment in which software runs must change over time. This causes "bit rot" as (say) dependencies stop receiving security updates or platforms become unsupported. Finally, there's the human factor. When software runs on old technologies, it will become progressively harder for new developers to understand. There will consequently often be a steady decline in available talent. For these reasons, even if the requirements of a piece of software never change, it will always require ongoing maintenance to keep it running, much like a piece of physical infrastructure. (edit: typo)

deleted 4y ago

[deleted]

wrp 4y ago

The attitude toward the development process expressed here is not uniformly accepted in all programming subcultures. There is a major cultural divide between embedded and server/desktop programming. My example is very old, but I had occasion to discuss software development with embedded systems engineers from Air Force projects in the pre-Ada days. They were all vehemently against the idea of software maintenance. They felt that their job was to deliver a "correct" implementation of a fully specified system.

smadge 4y ago
That model makes sense for aircraft, in my opinion. The software isn’t a standalone product that can be independently continuously improved. It’s tightly integrated with safety critical physical systems. You need to do the upfront work of specifying how the software and hardware interact and do the complex systems safety analysis. Any change to the software specification after design can have unintended effects on the overall behavior of the aircraft.

neilv 4y ago

> As Steve Jobs observed, 8 one of the major differences between hardware and software is that for hardware the “dynamic range” (ratio between the best in class and average performance) is, at most, 2:1. But, the difference between the best software developer and an average software developer can be 50:1, or even 100:1, and putting great developers on a team with other great developers amplifies this effect. Today, in DoD and the industrial base that supports it, the people with the necessary skills exist, but instead of taking advantage of their skills we put them in environments where it is difficult for them to be effective. DoD does not take advantage of already existing military and civilian personnel expertise by offering pay bonuses, career paths that provide the ability to stay in their specialization, or access to early promotions. Skilled software engineers and the related specialties that are part of the overall software ecosystem need to be treated as a special force; the United States must harness their talent for the great benefits that it can provide. As if we software people didn't have overinflated egos already... This DoD report talks about 100x developers existing (and of course HNers imagine ourselves towards that better-than-average end of the scale). The DoD here even uses the (lowercase) term "special force", which sounds like we're not nerd typists, but super-cool elite operators.

4648283 4y ago
Ironically I suspect that wind l while the idea will feed many egos the reality would crush or frustrate those same people. If the DoD certifies someone as a 100x developer it would be after crafting a ruthlessly banal evaluation system that would likely lack the romantic attitude towards software many developers have in my experience. Especially if the criteria included some social skills component that determined part of a developers rating by how well they synergize with similarly rated peers.
- neilv 4y ago
  I think the DoD has had many outstanding R&D successes, hiring or contracting elite engineers and scientists to do amazing things, which beat most of what we see from tech private sector B2C and B2B businesses. That's one place you'll need some people who can find brilliant solutions or insights that most people simply cannot, no matter whether you throw 100x times the number of people at the problem. But if they're having trouble attracting and retaining as many 1x-3x everyday software developers as they'd like, I agree, it'd be easy to imagine an attempt that fails miserably due to stereotypical bureaucracy. Paying close to FAANG TC levels would be a start, since it would cover the monetary factor, and i think the prestige factor would quickly follow. (I bet few people think adtech, fintech, or yet-another-microblogging/messaging/profile-app are actually cool anymore -- only the money is what makes those companies prestigious.) Maybe pay 80% of FAANG, so the FAANGs will still get the people who only want money, and you offer better non-monetary value to make up that 20%. (80% should be close enough they can afford to live in the same neighborhoods and send their kids to the same schools as FAANG, rather than be lower class that could upgrade just by grinding Leetcode for a few months.) Then manage those units like they are very serious excellent product development organizations where everyone wants to work. And figure out how to do the culture better than all the companies that got too much into class thinking, and too much jumping through the hoops for interview and pursuit of promotions, rather than focus on the mission and team.

killjoywashere 4y ago

The problem isn't that DoD requires waterfall for software. The problem is the Planning Programming Budgeting Execution cycle. If you solve a problem, you have at least an 18 month hole in the schedule. This drives PMs to waterfall so the can project funding out over the POM hole. It's profoundly broken and kept that way by Congress critters in the pocket of the prime integrators.

ngcc_hk 4y ago

What is meant by done? As part of system it co-evolves with it and due with it. It dies with the system it sort of “parasite” on or it is the key part of the system or it is the system. Still evolve until death.

einpoklum 4y ago

It's like reading a report by Darth Vader about the "national defense" of the galactic empire, explaining what's necessary for crushing "our adversaries" the rebels. Chill down my spine.

mihaigalos 4y ago

I feel Scrum has evolved into something contradicting the original Agile practices. These were more of a template on how to develop, not a concrete recipe. Scrum then established a process where there was need for a family of processes, continuously evolving, ever changing. The very fact that "you absolutely have to have a specific ceremony (i.e.: Retro when no topics)" is a red flag which imho puts the cart before the horse. Why is there a "Scrum Master" anyway, if individuals are encouraged to interact and own the process? Suppose an individual wants to change the process, and reaches consensus within the team/community. Why would a Scrum Master / Manager even be able to veto any decision? Why is there a Sprint when SW development is actually a marathon? Uncle Bob talks about this in several of his Clean Methodology books. I don't think that Agile is dead [1], although it has been taken over by people eager to sell you a *cough* certification (Scrum or any other). It's not dead because its genius is that it's constantly evolving, and you can always interpret it differently since it's only a template. Scrum, on the other hand.. well.. For the future, I look forward to not putting an equal sign between Scrum and Agile, even though apparently Scrum is how you "do" Agile, according to the industry. [1] https://www.simplethread.com/agile-at-20-the-failed-rebellion/ https://www.simplethread.com/agile-at-20-the-failed-rebellio...