5 ms·

It's a very 'agile' project. When will it be done? Don't ask! How much will it cost? Who knows! It's going to be awesome, though! But... you don't have to

by furyg3 12y ago

It's a very 'agile' project. When will it be done? Don't ask! How much will it cost? Who knows! It's going to be awesome, though! But... you don't have to wait to see how powerful this weapons system is. The F-35 completely took out the Dutch government before one even rolled off the assembly line! Note to governments: when ordering your weapons systems it may be a good idea to get a cost-per-unit quote before signing.

jacquesm 12y ago
> Note to governments: when ordering your weapons systems it may be a good idea to get a cost-per-unit quote before signing. What makes you think they're unhappy about the outcome?
- furyg3 12y ago
  Well considering parliament dissolved over the issue I can imagine there's a lot to be desired. However, like construction projects, the original politician/official get the credit for signing the contract and their successor actually has to deal with the terms.
  - hencq 12y ago
    I'm curious which government you think collapsed over the F-35. I can't think of any. Granted, there have been a lot of debates about the JSF program in Dutch politics, but unless I'm suffering from faulty memory none caused the government to collapse.
angdis 12y ago
A quote for something like this would be a waste of time. The "rules" are obviously different military hardware projects that have strategic significance: you either need it or you don't and if you have to ask "how much?", you can't afford it !
rayiner 12y ago
Defense systems go over-budget for the same reason software projects do--it's all custom and custom development is unpredictable. It's not like these companies can sell the U.S. stuff they had lying around having developed it for another customer. I used to do development for a DOD project. At the outset of the project, it's often not clear what you're trying to do will even work. It might not even be clear what you should be trying to do. It's an iterative process of the government telling the engineers what it wants, and the engineers going back to the government with what can be done. That's not the say that projects like the F-35 weren't badly managed. But even well-managed defense projects are unpredictable. The world just looks very different when you can't count on massive consumer volumes to amortize R&D. Imagine going to Apple and saying "we need you to develop the iPhone, and oh by the way we will order at most a few thousand of these. What'll be the unit cost?"
- enraged_camel 12y ago
  >>I used to do development for a DOD project. At the outset of the project, it's often not clear what you're trying to do will even work. It might not even be clear what you should be trying to do. That's interesting, and also scary. I work in IT consulting where it's extremely rare for customers to pay for a professional services project before seeing a proof of concept demonstration of what will be delivered. In fact, we insist on showing them one before anything is signed, because it aligns expectations and allows for more accurate SOWs.
  - harryh 12y ago
    It doesn't sound scary to me. It just sounds like the fundamental nature of trying to do something truly new. It's just less predictable and there is no way around that.
  - RogerL 12y ago
    Okay, extrapolate this to a weapons system. You can't expect a company to come up with their own version of the F-35 (which is extremely advanced) with the hope that, if you are satisfied, you'll buy 100 or so. If the government will not go at risk for this, there is no way a company, with vastly fewer resources, is going to do it. I've bid weapons systems and built weapons systems. I can honestly say that you make your very best effort to come up with a realistic proposal (we aren't trying to rip off the government. I'm just as much a taxpayer as you are). But there are limits. Detailed requirements are only going to be known well after you start. The government doesn't know how to specify the requirements - in large part that is your job. When you bid you want to be sure that you can be profitable, but then you also have competition, and a blind process (you don't know what your competitor is proposing, and at what price). This limits collusion, but the lack of information makes it hard to write a good proposal. Company A might think feature X is important, and put that in the contract. You didn't. Who is right? You'll find out when the contract is awarded. Opps if you got it wrong. Once you win the real "fun" begins. If you are lucky the program office running your contract understands the nature of research and the unknowability of the future, and works with you to set a reasonable scope for the project, and works with you as more information is found. OTOH you can have a bean counter trying to protect their career by counting "shalls" in the document (a legally binding term) and ensuring that each one can be checked off with what you deliver. Doesn't matter if it is rational or not. And then there are copious documentation requirements. The intent is that these requirements be tailored to the contract and needs, but again bean counters can intervene and have you doing massive amounts of busy work. A great program officer will stream line it and have you doing effective work. But, you know, there is not infinite money to prototype, build, test, scrap, and build again. You have 1 billion moving parts (figuratively) that you are trying to track and optimize for, and you kind of work your way to a solution. that is, unless you can afford the project to cost 10x as much (you are already complaining about cost, so that is a nonstarter), or take 5x as long (why don't I have my jet yet?!?!). The result is, unsurprisingly, unoptimal. tl;dr: weapons systems are an extremely expensive mix of state of the art research/development/deployment all in one. It is not going to go smoothly. It is the worst possible way to do a project, except for all the other ways, which are worse.
bunderbunder 12y ago
It's more a classic example of the situation that agile methodologies seek to avoid. The biggest thing Agile tries to do is minimize up-front planning. The reason for that is that you have the absolute least knowledge available to inform your decisions at the beginning of a project, so decisions made early on are massively more likely to be poor ones. Worse, you tend to get locked into them because other components in the system were designed around these decisions and that creates major impediments to change. F-35 had reams and reams of design decisions made long before anyone started assembling the thing. To an Agile team that's the worst possible thing you can do. You decide you're going to do X long before you have any idea how long X will take or what it will cost or how well it might work. OK sure, yes, you've written down some numbers on paper, but to be blunt that's just blowing smoke up your own ass. It's inevitable that the end result of so carefully specifying the requirements up-front would lead to a virtually unbounded schedule and budget. By contrast, an agile project's schedule and budget are theoretically unbounded, but you're supposed to strive to always be at a point where it's possible to ship on short notice so the practical reality (not always accomplished) should be that it's always possible to fix one or the other at a defined point by saying, "OK that's enough, let's ship." Or you can even pull the plug on the whole project if you decide it's just not working, and that's actually a realistic option because by delaying decisions you haven't encumbered yourself with a mountain of contracts and financial obligations. Which isn't to say you could get away with using Scrum or Kanban on a project like this. Just that the problems you're talking about aren't really emblematic of Agile so much as the problems Agile is designed to fix.
- Gravityloss 12y ago
  They did have the JSF competition with flying prototypes.
- RogerL 12y ago
  "move fast, break things" is not a viable approach to aircraft design. Lead times on hardware is years. That helmet - how long would it take you to make one? You can't just tell that team don't sweat it, we are going to change requirement on you every two weeks. And you can't tell the jet team wait 3 years for this helmet, then start designing the system. Military development is so far outside the realm of agile that it make so sense to compare (source: I've done both). The things agile people struggle with I just scoff at. Building large, physical, state of the art systems require planning, requires schedules, requires rework, requires huge teams, requires reams of documentation, and so on. You can't agile your way through that stuff. Planning doesn't lead to unbounded schedules. It leads to compromise. We have 3 years, 85 million dollars. We need an architecture (assume, say, a flight system for a helicopter, something I have done and delivered). I'm sorry, but very early in you have to choose your processors, distributed network (you need to be fail safe, so multiple redundancies are needed), and more. Then you can buy that, or simulate that; either has costs, but your engineers something to work with. Many have very strict environmental concerns - limited weight, has to fit in this bay, needs radiation hardening, and so on. I can't wait until the last 3 months to make this decision, sorry. Somebody has to build these, deliver them. They need to design and build out the logistics chain - how do I retrofit the existing fleet? HOw do I do repairs? How quickly can I spin up my factory line if you need spares? Do we buy 3x the units you need now, or plan on JIT. This is just one tiny nit of a tiny detail that needs to be decided and tracked. Don't forget instruction manuals, plans to train the maintaince workers, training the workers, training the pilots, building a school for the next people coming through in two years. I have a meething. I could type for several hours. There is nothing agile about this problem.
njharman 12y ago
Agile has fixed delivery date and variable features delivered on that date. Features are prioritized by customer at start of every iteration. Done right, Agile delivers exactly the things customer has chosen to afford. In my experience unfortunately, exactly 0% of customers (internal or external) are ablet to accept/wrap their head around a variable feature deliverable and demand a fix set of features decided at start and required to be delivered by some date. Aka utter fantasy.
- bunderbunder 12y ago
  You can do fixed delivery, variable features or fixed features, variable delivery. It doesn't matter which, the point is just to acknowledge that you can't fix both so you better understand which is more important to you. That said, it's perfectly reasonable for customers to expect to have some promise about what they're paying for. That's where you trot out the idea of a minimum viable product: Make some tough decisions about what features are actually required, and then agree to a timeline that makes it a virtual certainty that you'll be able to deliver all of the required ones. When you hit that mark ahead of schedule and start burning through the list of nice-to-haves they'll be elated.

by furyg3 12y ago

jacquesm 12y ago

> Note to governments: when ordering your weapons systems it may be a good idea to get a cost-per-unit quote before signing. What makes you think they're unhappy about the outcome?

furyg3 12y ago
Well considering parliament dissolved over the issue I can imagine there's a lot to be desired. However, like construction projects, the original politician/official get the credit for signing the contract and their successor actually has to deal with the terms.
- hencq 12y ago
  I'm curious which government you think collapsed over the F-35. I can't think of any. Granted, there have been a lot of debates about the JSF program in Dutch politics, but unless I'm suffering from faulty memory none caused the government to collapse.

angdis 12y ago

A quote for something like this would be a waste of time. The "rules" are obviously different military hardware projects that have strategic significance: you either need it or you don't and if you have to ask "how much?", you can't afford it !

rayiner 12y ago

Defense systems go over-budget for the same reason software projects do--it's all custom and custom development is unpredictable. It's not like these companies can sell the U.S. stuff they had lying around having developed it for another customer. I used to do development for a DOD project. At the outset of the project, it's often not clear what you're trying to do will even work. It might not even be clear what you should be trying to do. It's an iterative process of the government telling the engineers what it wants, and the engineers going back to the government with what can be done. That's not the say that projects like the F-35 weren't badly managed. But even well-managed defense projects are unpredictable. The world just looks very different when you can't count on massive consumer volumes to amortize R&D. Imagine going to Apple and saying "we need you to develop the iPhone, and oh by the way we will order at most a few thousand of these. What'll be the unit cost?"

enraged_camel 12y ago
>>I used to do development for a DOD project. At the outset of the project, it's often not clear what you're trying to do will even work. It might not even be clear what you should be trying to do. That's interesting, and also scary. I work in IT consulting where it's extremely rare for customers to pay for a professional services project before seeing a proof of concept demonstration of what will be delivered. In fact, we insist on showing them one before anything is signed, because it aligns expectations and allows for more accurate SOWs.
- harryh 12y ago
  It doesn't sound scary to me. It just sounds like the fundamental nature of trying to do something truly new. It's just less predictable and there is no way around that.
- RogerL 12y ago
  Okay, extrapolate this to a weapons system. You can't expect a company to come up with their own version of the F-35 (which is extremely advanced) with the hope that, if you are satisfied, you'll buy 100 or so. If the government will not go at risk for this, there is no way a company, with vastly fewer resources, is going to do it. I've bid weapons systems and built weapons systems. I can honestly say that you make your very best effort to come up with a realistic proposal (we aren't trying to rip off the government. I'm just as much a taxpayer as you are). But there are limits. Detailed requirements are only going to be known well after you start. The government doesn't know how to specify the requirements - in large part that is your job. When you bid you want to be sure that you can be profitable, but then you also have competition, and a blind process (you don't know what your competitor is proposing, and at what price). This limits collusion, but the lack of information makes it hard to write a good proposal. Company A might think feature X is important, and put that in the contract. You didn't. Who is right? You'll find out when the contract is awarded. Opps if you got it wrong. Once you win the real "fun" begins. If you are lucky the program office running your contract understands the nature of research and the unknowability of the future, and works with you to set a reasonable scope for the project, and works with you as more information is found. OTOH you can have a bean counter trying to protect their career by counting "shalls" in the document (a legally binding term) and ensuring that each one can be checked off with what you deliver. Doesn't matter if it is rational or not. And then there are copious documentation requirements. The intent is that these requirements be tailored to the contract and needs, but again bean counters can intervene and have you doing massive amounts of busy work. A great program officer will stream line it and have you doing effective work. But, you know, there is not infinite money to prototype, build, test, scrap, and build again. You have 1 billion moving parts (figuratively) that you are trying to track and optimize for, and you kind of work your way to a solution. that is, unless you can afford the project to cost 10x as much (you are already complaining about cost, so that is a nonstarter), or take 5x as long (why don't I have my jet yet?!?!). The result is, unsurprisingly, unoptimal. tl;dr: weapons systems are an extremely expensive mix of state of the art research/development/deployment all in one. It is not going to go smoothly. It is the worst possible way to do a project, except for all the other ways, which are worse.

bunderbunder 12y ago

It's more a classic example of the situation that agile methodologies seek to avoid. The biggest thing Agile tries to do is minimize up-front planning. The reason for that is that you have the absolute least knowledge available to inform your decisions at the beginning of a project, so decisions made early on are massively more likely to be poor ones. Worse, you tend to get locked into them because other components in the system were designed around these decisions and that creates major impediments to change. F-35 had reams and reams of design decisions made long before anyone started assembling the thing. To an Agile team that's the worst possible thing you can do. You decide you're going to do X long before you have any idea how long X will take or what it will cost or how well it might work. OK sure, yes, you've written down some numbers on paper, but to be blunt that's just blowing smoke up your own ass. It's inevitable that the end result of so carefully specifying the requirements up-front would lead to a virtually unbounded schedule and budget. By contrast, an agile project's schedule and budget are theoretically unbounded, but you're supposed to strive to always be at a point where it's possible to ship on short notice so the practical reality (not always accomplished) should be that it's always possible to fix one or the other at a defined point by saying, "OK that's enough, let's ship." Or you can even pull the plug on the whole project if you decide it's just not working, and that's actually a realistic option because by delaying decisions you haven't encumbered yourself with a mountain of contracts and financial obligations. Which isn't to say you could get away with using Scrum or Kanban on a project like this. Just that the problems you're talking about aren't really emblematic of Agile so much as the problems Agile is designed to fix.

Gravityloss 12y ago
They did have the JSF competition with flying prototypes.
RogerL 12y ago
"move fast, break things" is not a viable approach to aircraft design. Lead times on hardware is years. That helmet - how long would it take you to make one? You can't just tell that team don't sweat it, we are going to change requirement on you every two weeks. And you can't tell the jet team wait 3 years for this helmet, then start designing the system. Military development is so far outside the realm of agile that it make so sense to compare (source: I've done both). The things agile people struggle with I just scoff at. Building large, physical, state of the art systems require planning, requires schedules, requires rework, requires huge teams, requires reams of documentation, and so on. You can't agile your way through that stuff. Planning doesn't lead to unbounded schedules. It leads to compromise. We have 3 years, 85 million dollars. We need an architecture (assume, say, a flight system for a helicopter, something I have done and delivered). I'm sorry, but very early in you have to choose your processors, distributed network (you need to be fail safe, so multiple redundancies are needed), and more. Then you can buy that, or simulate that; either has costs, but your engineers something to work with. Many have very strict environmental concerns - limited weight, has to fit in this bay, needs radiation hardening, and so on. I can't wait until the last 3 months to make this decision, sorry. Somebody has to build these, deliver them. They need to design and build out the logistics chain - how do I retrofit the existing fleet? HOw do I do repairs? How quickly can I spin up my factory line if you need spares? Do we buy 3x the units you need now, or plan on JIT. This is just one tiny nit of a tiny detail that needs to be decided and tracked. Don't forget instruction manuals, plans to train the maintaince workers, training the workers, training the pilots, building a school for the next people coming through in two years. I have a meething. I could type for several hours. There is nothing agile about this problem.

njharman 12y ago

Agile has fixed delivery date and variable features delivered on that date. Features are prioritized by customer at start of every iteration. Done right, Agile delivers exactly the things customer has chosen to afford. In my experience unfortunately, exactly 0% of customers (internal or external) are ablet to accept/wrap their head around a variable feature deliverable and demand a fix set of features decided at start and required to be delivered by some date. Aka utter fantasy.

bunderbunder 12y ago
You can do fixed delivery, variable features or fixed features, variable delivery. It doesn't matter which, the point is just to acknowledge that you can't fix both so you better understand which is more important to you. That said, it's perfectly reasonable for customers to expect to have some promise about what they're paying for. That's where you trot out the idea of a minimum viable product: Make some tough decisions about what features are actually required, and then agree to a timeline that makes it a virtual certainty that you'll be able to deliver all of the required ones. When you hit that mark ahead of schedule and start burning through the list of nice-to-haves they'll be elated.