8 ms·

Do we have any evidence that Agile actually helps, and for what kind of projects / companies it helps? It's just I note this is a DOD document, and I don't thi

by shubb 8y ago

Do we have any evidence that Agile actually helps, and for what kind of projects / companies it helps? It's just I note this is a DOD document, and I don't think we should be desiging jet fighters in 2 week sprints. Airframes probably can't be modified like that. So it follows that, within a waterfall project, the flight software shouldn't be developed agile either. Beyond that, I don't think safety critical systems should be produced code first. We should probably be doing modelling and formal verification and stuff like that. I think agile is a great way of making e commerce websites. Sure, XP came out of NASA, but XP is a specific thing. Not working overtime is important in XP. 'Modern Agile' on the other hand cares about things like producing an MVP and iterating, which originated in and are appropriate in a web startup context. I'm not sure they should be forced in an engineering context.

Kalium 8y ago
Even the DoD has plenty of fairly generic webapps it needs developed and maintained. I worked on one that was "agile" - it definitely ran afoul of some of these pointers.
- t0mbstone 8y ago
  Well, yeah... I would imagine that's why this document exists and they are attempting to push it for the 2019 and 2020 agenda
fouc 8y ago
Presumably Agile is for software projects, not hardware.
- howard941 8y ago
  IME this is not a good presumption. I'm aware of its use at one local manufacturer of electronic medical devices, another that makes fire protection systems, and another that makes those ankle bracelet tracking things.
mi100hael 8y ago
> I don't think we should be desiging jet fighters in 2 week sprints. Airframes probably can't be modified like that. So it follows that, within a waterfall project, the flight software shouldn't be developed agile either. Do you have any basis for that position? I think using agile principles (ie making small modifications and testing them frequently to get feedback) seems like a great way to design a complicated piece of machinery.
- JoeAltmaier 8y ago
  Its a great way to churn through hundreds of small tasks, like hooking up database queries to web pages or changing graphics on a site. Its a terrible, terrible way to try to design complex control systems. They must be studied and modelled and analyzed by experts in processes that have absolutely nothing to do with '2 week sprints'. =
  - lgunsch 8y ago
    What is terrible about iteratively refining a complex design as you gain feedback from its implementation?
    - AnimalMuppet 8y ago
      Stronger: What's worse about "iteratively refining a complex design as you gain feedback from its implementation", compared to "not iteratively refining a complex design as you gain feedback from its implementation"?
    - JoeAltmaier 8y ago
      And what does {what you said} have to do with 2-week sprints?
  - SamuelAdams 8y ago
    By the time developers are working on the change, the design should already be complete. An assembly line worker isn't going to be considering the design of a car: they are going to add this one part according to the blueprint, and move on. They trust the engineers / architects' plan, which should have already been done by the time they started building.
commandlinefan 8y ago
> Do we have any evidence that Agile actually helps In my experience (25+ years now), an awful lot of people who are looking into adopting Agile processes are actually in the midst of grasping at straws/looking for a silver bullet. The "problem" they have is that software development takes longer than they want it to and no matter how hard they beat the programmers, it keeps taking longer than they want it to. So "agile" is just the newest cudgel.
- alexeiz 8y ago
  > no matter how hard they beat the programmers Maybe they should try not beating the programmers. I'm just saying...
  - fein 8y ago
    The beatings will continue until morale improves.
- dimgl 8y ago
  This is exactly right. I was at two companies in Florida who were "agile" and it was extremely pathetic. Daily standups, marching orders, insane requirements and deadlines, overly long meetings, tons of PMs and 0 automated/QA testing. In fact, they were hiring more QA personnel and project managers at both companies and keeping the dev count the same. So yeah... This is typical stuff.
ionforce 8y ago
> I don't think we should be desiging jet fighters in 2 week sprints. Who is making this claim?
- trhway 8y ago
  Not even fighter jets. When several years ago our BigCo implemented total Agile Scrum (only few super important super critical projects got exception) with the best consultants one can get for the money the result was exactly as could be easily predicted by applying basic system analysis to those methodologies - the projects halted to the screeching halts while people and organizations were spinning at FTL speeds with everything just burning around on contact. I've never seen such profound yet so easily predictable failures. One magical day the consultants and all and any mentioning of agile, scrum, etc just disappeared like it never happened, just like a bad dream, and we woke up and smelling roses again... Any meaningful progress on a project subjected to agile/scrum is possible only to the extent to which the project violates the agile/scrum process. Of course you're very welcome to tell me that we weren't doing Agile/Scrum right...
  - closeparen 8y ago
    So what were you doing?
  - parasubvert 8y ago
    Is it possible to build software successfully with agile processes? Yes. Is it possible to build software successfully without agile processes? Also yes. The point is that, blaming agile for your failure is about as pointless as saying agile is a saviour. Any corporate forensic anthropologist would need to look at a lot of process, incentive, and organizational specifics to know what was and was not working in either case.
    - trhway 8y ago
      >Is it possible to build software successfully with agile processes? Yes. not really. Until of course your software is simple - ie. 1. doesn't have any significant internal nor external dependencies being developed as part of the same project/during the same time and 2. any feature can be done by a one "full stack developer". Agile/Scrum don't provide for dependencies management, and when implemented thorough and correctly the Agile/Scrum actively and severely impedes dependencies management. >The point is that, blaming agile for your failure is about as pointless as saying agile is a saviour. I'm not blaming Agile. The Agile in our case of various mildly to really complex enterprise software projects worked exactly as designed - ie. practically halted down any progress of the projects. I'm blaming the people who decided to implement the Agile/Scrum in our company when it is an obviously unsuitable process for any minimally complex projects. Besides the above mentioned dependency management issue, the Agile/Scrum main effort is directed toward dramatically lowering of the latencies in the dev process (which main effect is totally transparent always up-to-date status - management's wet dream, and boy, did we have it!) while promising bandwidth/throughput (productivity) increase at the same cost (i.e. the same developers/teams). The fallacy of "it is possible to pick all the 3 at the same time" - the lower latency and increased bandwidth/throughput while keeping the same cost/hardware - would be easily recognized by any normal engineer in the context of a normal engineering problem. Yet when it comes to process management, many otherwise reasonable engineers do unexplainably believe in the pixie dust of Agile/Scrum. When somebody says "we're successful and we're using Agile/Scrum" it has always (until it is 10x dev case described below) been the case in my experience that hey do have significant deviations from a complete thorough Agile/Scrum implementation and those deviations provide the "breathing windows" which allow for their survival. Another situation of "we're successful and we're using Agile/Scrum" is that the companies employing "10x" developers, like Google, FB and many well-funded hip startups can allow themselves to decimate the engineers' performance/bandwidth/throughput even by say 5 times (by the combination of factors like open floor office, Agile/Scrum process management, etc.) and they would still be left with effectively "2x" performing developers which still allows to be pretty successful. Similar scale decimation of performance/throughput of a typical "1x" developer at a typical BigCo like say ours naturally leaves you with "1x/5" :)
pjc50 8y ago
> I don't think we should be desiging jet fighters in 2 week sprints. Airframes probably can't be modified like that The F35 might be the first plane in history where the software costs more than the airframe development. It's difficult to find a detailed breakdown, but example numbers include https://www.theregister.co.uk/2018/03/22/f_35b_block_4_upgrades_cost_uk_345m/ https://www.theregister.co.uk/2018/03/22/f_35b_block_4_upgra... "Figures reported by The Times newspaper pegged US estimates for upgrades to the supersonic stealth fighter at £7.67bn ($10.8bn) for software development and £3.84bn ($5.4bn) for deploying those upgrades across all F-35s ever built." And that's just an upgrade. Now, arguably you don't want to be too agile with the avionics either, but at some point the cost of not delivering anything that works becomes prohibitive.
- shubb 8y ago
  I'm not clear which way you are arguing here. An upgrade is additonal work right? Like, an extra sprint. It turned out to be crazy expensive. The F35 programme went wrong because they built an MVP (F35A) and then decided to derive 2 extra planes from that design (F35 B and C). Turned out that modifying an existing air frame to make it VTOL is really hard and they would have been better off designing a new plane from scratch. As you say we weren't there, don't have the data, and judging the facts from a noteably freakishly unsuccesful programme isn't sensible but... I mean, I think it supports the idea we shouldn't do this.
- deleted 8y ago
  [deleted]
Someone 8y ago
”I don't think we should be desiging jet fighters in 2 week sprints” Then make the sprints longer. IMO, the Apollo project was run in an agile fashion (“They want us to send men to the moon, but we can’t go there, nor send men up. What baby step can we make instead?”)
- jhayward 8y ago
  There is a difference between iterative or incremental development and agile. Iterative processes have been used for 70-80 years. The first example on the wiki page [1] is the Mercury rocket program. [1] https://en.wikipedia.org/wiki/Iterative_and_incremental_development#Usage/History https://en.wikipedia.org/wiki/Iterative_and_incremental_deve...
  - discreteevent 8y ago
    There is difference alright but I think IID and a small team that meet regularly is all you need.
  - parasubvert 8y ago
    Agile is a set of practices intended to shorten the feedback cycle of “increments and iterations”, which became ineffectively long and unwieldy in both commercial and defense, perhaps in part due to a greater value placed on political perception of a program rather than its actual results.
starpilot 8y ago
I felt like Agile was like rewarding people for rats tails, so people start breeding rats. There was too much focus on clearing out stories, even if it meant a ton of bugs, but the bugs were good because those led to more stories and the PM could take credit for "getting so much done." This was at a certain dysfunctional, highly political networking hardware company.
- nradov 8y ago
  That type of dysfunction can occur under any methodology, regardless of agility. It indicates an inadequate definition of done and poor quality procedures.
- aaaaaaaaaab 8y ago
  So that’s why all network gear needs to be power-cycled at least weekly to function correctly!
  - outworlder 8y ago
    No, although that could be part of it. You are probably talking about consumer gear. Consumers usually have two requirements: a) FEATURES FEATURES FEATURES. All of them. 200 beamforming antennas. Interplanetary internet. b) Price. All of the above for the lowest price possible. FTL communication? Cool. It better be under $50 or I'll buy the competitor. So this is what the companies optimize for. Who cares if DHCP is broken if it works most of the time. Memory leak? Just have it reboot. IPV6 broken? Most customers won't even know or care. You'll find that 'enterprise' hardware performs way better. At a price premium.
- Jtsummers 8y ago
  That's not agile. That's goal-driven motivation systems. "Finish twice the average stories per month and you get a bonus!" People game the rewards system by generating smaller, achievable stories. I had a colleague once who [0] would save up test cases (we were on the V&V side) to develop that were smaller. He could've had them all done and delivering value in two weeks, 200 test cases done! But if he did that he'd be left with the remaining 50 test cases that each needed 1-3 weeks to develop test systems for (sometimes shorter, I think 2 weeks or so was the average). So if he found himself looking bad on the burndown chart, he'd churn out 20 finished test cases from the easy ones. This way he was always "on track". If he'd done them and gotten ahead, they would've wondered why he slowed down (because not all test cases had the same size, duh) and penalized him for it. The problem is, that goals like that (as carrot or stick) don't change the fundamental system. If I manage to churn out 20 story points this week, but only 10 every other week, is it sustainable? Was it a fluke? Was it because I did 40 hours of OT? Or was it that they were overestimated? Maybe I had a unique skill set or insight that let me solve it more quickly than had been anticipated. Maybe something in my brain clicked (remember that moment early in your career where a "hard" concept suddenly became obvious, and all your programming tasks became much easier?) and the new level will be sustainable. On the other hand, if you create a goal to double efficiency/capacity and provide access to resources to change the development/deployment/whatever system then you can get something useful. I wrote a tool for data analysis for a project (actually a collection of related projects). The tool cut down test analysis time from hours to minutes. An activity done daily by the V&V team. I saved them years of work in that first year. The goal was to reduce the bottleneck (test results took too long to get so developers were waiting or operating blind). That goal, by itself, was worthless. It was only able to change the system of that team because I had been given permission to work on tools that would achieve it. [0] We had a manager who did things like this, though no carrot all stick.
nradov 8y ago
Agile development practices are now more relevant to the early stages of airframe design when no real hardware exists and all testing is done in computer simulations. At that point changes are relatively cheap and there are no safety concerns. But once the outer mold line is set and prototype construction starts the engineers must be far more cautious about design changes.
regularfry 8y ago
This is one of those things that sounds true, but isn't. The Saab Gripen was designed with Scrum: http://www.agilitycode.com/agile-best-practices/owning-the-sky-with-agile-saabs-jas39e-gripen-fighter-plane-building-a-jet-better-faster-cheaper-with-scrum/ http://www.agilitycode.com/agile-best-practices/owning-the-s...
SolarNet 8y ago
I mean I don't think this document is actually taking a stance on things being agile. Just on the detection of things that are actually agile and those that aren't due to the prevalence of the buzzword.
peterwwillis 8y ago
If you've ever worked in a giant industrial organization, you know it's a swamp of red tape, silos, and inefficiency. Agile is a direct response to all this. Scrum, which influenced Agile, originated from case studies of automotive, photocopier and printer industries. Agile is not 2 week sprints. Agile is not code reviews. Agile is not pair programming. Agile is abstract; it's a set of values, a philosophy, a methodology. It screams, "Stop just following some stupid formula, because it may not work for you!" The whole point is to shake up the slow, rigid, inefficient way that organizations can easily fall into, and get people to work together better. If you take the guidelines it offers and apply them, any organization can benefit. But you can't just follow a formula and be magically better. What I've learned is that all of the abstract methodologies (agile, lean, devops, etc) do not prescribe specific actions you must take; they prescribe general ways that you can adapt your work to get better outcomes. I can't give you exact evidence that it works, because there is no one way that it works. But there are tons of organizations that have followed the principles and have measurably improved their businesses. Until someone turns all of business into a science, I don't think you'll find anything better.
- koonsolo 8y ago
  Exactly this. The problem that people have with Agile are mostly because they think they need to follow rigid Agile rules. For me, there is only 1 true rule in Agile: good retrospectives! Once you have good retrospectives, you can improve anything within the process. Any argument about "We cannot do 2 week sprints" falls out, because your team can decide whether 2 week sprints is in your best interest.
  - peterwwillis 8y ago
    Another way to refer to retrospectives would be as "Process Kaizen" (https://en.wikipedia.org/wiki/Kaizen https://en.wikipedia.org/wiki/Kaizen), or "PDCA" (https://en.wikipedia.org/wiki/PDCA https://en.wikipedia.org/wiki/PDCA). Both can be used to continuously improve pretty much anything.
wenc 8y ago
> We should probably be doing modelling and formal verification and stuff like that. This is interesting. Is this a hypothetical notion or can you share any personal experience of having worked on any projects where this is done successfully? I bring this up because there is typically a narrow band of applicability of purely theoretical constructs in most complex systems (AWS & Azure teams both use formal methods [1] to find bugs and to increase confidence in critical algorithms but I don't believe they do this system-wide. Also the verification is only done on the abstraction, not the actual implementation.). Richard Cook gave a talk on "How Complex Systems Fail" [2] which addresses this point, and advocates for designing for resiliency as opposed to designing to exhaustively handle all failure modes (which are impossible to predict). Design people often design from some idealized notion of inputs and/or worst case scenarios. Ops people know what can go wrong will go wrong, which is why ops is there to handle the deviations from the norm. Real world design exposes lift points, reveals controls and supports continuous maintenance. Highly reliable systems are evolved, not just designed. [1] https://en.wikipedia.org/wiki/TLA%2B#Industry_use https://en.wikipedia.org/wiki/TLA%2B#Industry_use [2] https://www.youtube.com/watch?v=2S0k12uZR14 https://www.youtube.com/watch?v=2S0k12uZR14
- jimbokun 8y ago
  "Richard Cook gave a talk on "How Complex Systems Fail" [2] which addresses this point, and advocates for designing for resiliency as opposed to designing to exhaustively handle all failure modes (which are impossible to predict)." This is the Erlang design philosophy.
jnwatson 8y ago
I'm not exactly a huge agile fan, but there are a few things it gets right. Most of the time, software development is implicitly a process of requirements development. Nobody ever knows what they want until they see it, even if they claim they do. Agile attempts to turn the implicit requirements development into explicit requirements development. It tries to shortcut the process of acquiring requirements by getting stuff in front of the user as fast as possible. Now, if you really do know the requirements (like there's a real spec to follow) or you're in a situation where you don't have the architecture decided so that you can show discrete things in a small time window (2-3 weeks), then agile isn't nearly as useful.
Jtsummers 8y ago
> 'Modern Agile' on the other hand cares about things like producing an MVP and iterating, which originated in and are appropriate in a web startup context. [emphasis added] That's a damned lie. Iterative and spiral development greatly pre-date web development. They pre-date the Web itself for software (though perhaps not the Internet, or at least Arpanet). Iterative and spiral development is the only sane way to develop any complex system (or at least to develop its specification). Agile is a further extension of that whose proponents further work to breakdown the knowledge and communication barriers that inhibit effective project development and management.
tacon 8y ago
>It's just I note this is a DOD document, and I don't think we should be designing jet fighters in 2 week sprints. Airframes probably can't be modified like that. So it follows that, within a waterfall project, the flight software shouldn't be developed agile either. There are some linked documents at the end of the OP, and this one with link "to be added"[0]. It addresses your exact point, in that hardware development and software development have to move in completely different processes. Their first example is the DoD 5000 process[1] and how it is a total fail for software: "The DoD 5000 process , depicted on the left, provides a detailed DoD process for setting requirements for complex systems and ensuring that delivered systems are compliant with those requirements. The DoD’s “one size fits all” approach to acquisition has attempted to apply this model to software systems, where it is wholly inappropriate. Software is different than hardware. Modern software methods make use of a much more iterative process, often referred to as “DevOps,” in which development and deployment (operations) are a continuous process, as depicted on the right. A key aspect of DevOps is continuous delivery of improved functionality through interaction with the end user." "DoD 5000 is designed to give OSD, the Services, and Congress some level of visibility and oversight into the development, acquisition, and sustainment of large weapons systems. While this directive may be useful for weapons systems with multi-billion dollar unit costs, it does not make sense for most software systems." [0] https://media.defense.gov/2018/Nov/02/2002058905/-1/-1/0/DIB_DOS_DONTS_%20SOFTWARE_V2_2018.11.02.PDF https://media.defense.gov/2018/Nov/02/2002058905/-1/-1/0/DIB... [1] http://acqnotes.com/wp-content/uploads/2014/09/DoD-Instruction-5000.02-The-Defense-Acquisition-System-10-Aug-17-Change-3.pdf http://acqnotes.com/wp-content/uploads/2014/09/DoD-Instructi...
myth2018 8y ago
I ask myself some similar questions. I worked for a huge Brazilian ERP company. Three years ago the board decided that the company was not realizing its full potential (which I agree with) and decided to give Agile a try. Since then, some really critical areas started to suffer a lot, the latest version is being heavily criticized by the users and the company's reputation is consistently going down. The answer seems obvious: they are doing Agile wrong, they missed the point entirely, the "big company" mindset is hindering the efforts and so on. This is, in fact, how the whole company is behaving about the poor results so far, treating any suggestion towards a Agile's rollback as a cabal proof of the company's inability in delivering software the proper way. But I dare to think that Agile might not be the only path. Most of the companies around are delivering fairly complex projects employing small and efficient and competent teams and this is really great. But some companies are not that lucky. Somebody has to build that monstrous stuff over two-decade-old legacy code foundations, employing many dozens of teams. You simply can't afford hiring the best people around, because they are not available nor willing to work for you and, anyway, there aren't enough of them. You can't work with the best users, because you have no control at all over their teams and they aren't too excited about you bothering them with your project -- and this is bad because users in Agile projects are a critical part of the team. You may somehow enforce them to work with you and this has already been made many times, but the resultant half-hearted participations are not worth the hassle. There are so many practical issues regarding a real-world agile culture implementation, so many out-of-control environmental variables affecting you very negatively, and they all seem to end up being buried under a set of principles which have allegedly worked somewhere else. As if it were easy, or even desirable, to blame your context for not being "mature enough" to adopt Agile. I've never seriously looked into it, but I believe that there are some rigorous academic case studies discussing cases where agile didn't work well. Although I believe they are scarce, both because Agile tend to fail in a smaller, yet important, set of projects, and also because they would hurt some strong beliefs which are currently being taken as common-sense. I also believe their analysis may occasionally show some bias too, perhaps focusing more on the "mistakes" made by the teams instead of considering the hypothesis of Agile being inappropriate in the analyzed case.

by shubb 8y ago

Kalium 8y ago

Even the DoD has plenty of fairly generic webapps it needs developed and maintained. I worked on one that was "agile" - it definitely ran afoul of some of these pointers.

t0mbstone 8y ago
Well, yeah... I would imagine that's why this document exists and they are attempting to push it for the 2019 and 2020 agenda

fouc 8y ago

Presumably Agile is for software projects, not hardware.

howard941 8y ago
IME this is not a good presumption. I'm aware of its use at one local manufacturer of electronic medical devices, another that makes fire protection systems, and another that makes those ankle bracelet tracking things.

mi100hael 8y ago

> I don't think we should be desiging jet fighters in 2 week sprints. Airframes probably can't be modified like that. So it follows that, within a waterfall project, the flight software shouldn't be developed agile either. Do you have any basis for that position? I think using agile principles (ie making small modifications and testing them frequently to get feedback) seems like a great way to design a complicated piece of machinery.

JoeAltmaier 8y ago
Its a great way to churn through hundreds of small tasks, like hooking up database queries to web pages or changing graphics on a site. Its a terrible, terrible way to try to design complex control systems. They must be studied and modelled and analyzed by experts in processes that have absolutely nothing to do with '2 week sprints'. =
- lgunsch 8y ago
  What is terrible about iteratively refining a complex design as you gain feedback from its implementation?
  - AnimalMuppet 8y ago
    Stronger: What's worse about "iteratively refining a complex design as you gain feedback from its implementation", compared to "not iteratively refining a complex design as you gain feedback from its implementation"?
  - JoeAltmaier 8y ago
    And what does {what you said} have to do with 2-week sprints?
- SamuelAdams 8y ago
  By the time developers are working on the change, the design should already be complete. An assembly line worker isn't going to be considering the design of a car: they are going to add this one part according to the blueprint, and move on. They trust the engineers / architects' plan, which should have already been done by the time they started building.

commandlinefan 8y ago

> Do we have any evidence that Agile actually helps In my experience (25+ years now), an awful lot of people who are looking into adopting Agile processes are actually in the midst of grasping at straws/looking for a silver bullet. The "problem" they have is that software development takes longer than they want it to and no matter how hard they beat the programmers, it keeps taking longer than they want it to. So "agile" is just the newest cudgel.

alexeiz 8y ago
> no matter how hard they beat the programmers Maybe they should try not beating the programmers. I'm just saying...
- fein 8y ago
  The beatings will continue until morale improves.
dimgl 8y ago
This is exactly right. I was at two companies in Florida who were "agile" and it was extremely pathetic. Daily standups, marching orders, insane requirements and deadlines, overly long meetings, tons of PMs and 0 automated/QA testing. In fact, they were hiring more QA personnel and project managers at both companies and keeping the dev count the same. So yeah... This is typical stuff.

ionforce 8y ago

> I don't think we should be desiging jet fighters in 2 week sprints. Who is making this claim?

trhway 8y ago
Not even fighter jets. When several years ago our BigCo implemented total Agile Scrum (only few super important super critical projects got exception) with the best consultants one can get for the money the result was exactly as could be easily predicted by applying basic system analysis to those methodologies - the projects halted to the screeching halts while people and organizations were spinning at FTL speeds with everything just burning around on contact. I've never seen such profound yet so easily predictable failures. One magical day the consultants and all and any mentioning of agile, scrum, etc just disappeared like it never happened, just like a bad dream, and we woke up and smelling roses again... Any meaningful progress on a project subjected to agile/scrum is possible only to the extent to which the project violates the agile/scrum process. Of course you're very welcome to tell me that we weren't doing Agile/Scrum right...
- closeparen 8y ago
  So what were you doing?
- parasubvert 8y ago
  Is it possible to build software successfully with agile processes? Yes. Is it possible to build software successfully without agile processes? Also yes. The point is that, blaming agile for your failure is about as pointless as saying agile is a saviour. Any corporate forensic anthropologist would need to look at a lot of process, incentive, and organizational specifics to know what was and was not working in either case.
  - trhway 8y ago
    >Is it possible to build software successfully with agile processes? Yes. not really. Until of course your software is simple - ie. 1. doesn't have any significant internal nor external dependencies being developed as part of the same project/during the same time and 2. any feature can be done by a one "full stack developer". Agile/Scrum don't provide for dependencies management, and when implemented thorough and correctly the Agile/Scrum actively and severely impedes dependencies management. >The point is that, blaming agile for your failure is about as pointless as saying agile is a saviour. I'm not blaming Agile. The Agile in our case of various mildly to really complex enterprise software projects worked exactly as designed - ie. practically halted down any progress of the projects. I'm blaming the people who decided to implement the Agile/Scrum in our company when it is an obviously unsuitable process for any minimally complex projects. Besides the above mentioned dependency management issue, the Agile/Scrum main effort is directed toward dramatically lowering of the latencies in the dev process (which main effect is totally transparent always up-to-date status - management's wet dream, and boy, did we have it!) while promising bandwidth/throughput (productivity) increase at the same cost (i.e. the same developers/teams). The fallacy of "it is possible to pick all the 3 at the same time" - the lower latency and increased bandwidth/throughput while keeping the same cost/hardware - would be easily recognized by any normal engineer in the context of a normal engineering problem. Yet when it comes to process management, many otherwise reasonable engineers do unexplainably believe in the pixie dust of Agile/Scrum. When somebody says "we're successful and we're using Agile/Scrum" it has always (until it is 10x dev case described below) been the case in my experience that hey do have significant deviations from a complete thorough Agile/Scrum implementation and those deviations provide the "breathing windows" which allow for their survival. Another situation of "we're successful and we're using Agile/Scrum" is that the companies employing "10x" developers, like Google, FB and many well-funded hip startups can allow themselves to decimate the engineers' performance/bandwidth/throughput even by say 5 times (by the combination of factors like open floor office, Agile/Scrum process management, etc.) and they would still be left with effectively "2x" performing developers which still allows to be pretty successful. Similar scale decimation of performance/throughput of a typical "1x" developer at a typical BigCo like say ours naturally leaves you with "1x/5" :)

pjc50 8y ago

> I don't think we should be desiging jet fighters in 2 week sprints. Airframes probably can't be modified like that The F35 might be the first plane in history where the software costs more than the airframe development. It's difficult to find a detailed breakdown, but example numbers include https://www.theregister.co.uk/2018/03/22/f_35b_block_4_upgrades_cost_uk_345m/ https://www.theregister.co.uk/2018/03/22/f_35b_block_4_upgra... "Figures reported by The Times newspaper pegged US estimates for upgrades to the supersonic stealth fighter at £7.67bn ($10.8bn) for software development and £3.84bn ($5.4bn) for deploying those upgrades across all F-35s ever built." And that's just an upgrade. Now, arguably you don't want to be too agile with the avionics either, but at some point the cost of not delivering anything that works becomes prohibitive.

shubb 8y ago
I'm not clear which way you are arguing here. An upgrade is additonal work right? Like, an extra sprint. It turned out to be crazy expensive. The F35 programme went wrong because they built an MVP (F35A) and then decided to derive 2 extra planes from that design (F35 B and C). Turned out that modifying an existing air frame to make it VTOL is really hard and they would have been better off designing a new plane from scratch. As you say we weren't there, don't have the data, and judging the facts from a noteably freakishly unsuccesful programme isn't sensible but... I mean, I think it supports the idea we shouldn't do this.
deleted 8y ago
[deleted]

Someone 8y ago

”I don't think we should be desiging jet fighters in 2 week sprints” Then make the sprints longer. IMO, the Apollo project was run in an agile fashion (“They want us to send men to the moon, but we can’t go there, nor send men up. What baby step can we make instead?”)

jhayward 8y ago
There is a difference between iterative or incremental development and agile. Iterative processes have been used for 70-80 years. The first example on the wiki page [1] is the Mercury rocket program. [1] https://en.wikipedia.org/wiki/Iterative_and_incremental_development#Usage/History https://en.wikipedia.org/wiki/Iterative_and_incremental_deve...
- discreteevent 8y ago
  There is difference alright but I think IID and a small team that meet regularly is all you need.
- parasubvert 8y ago
  Agile is a set of practices intended to shorten the feedback cycle of “increments and iterations”, which became ineffectively long and unwieldy in both commercial and defense, perhaps in part due to a greater value placed on political perception of a program rather than its actual results.

starpilot 8y ago

I felt like Agile was like rewarding people for rats tails, so people start breeding rats. There was too much focus on clearing out stories, even if it meant a ton of bugs, but the bugs were good because those led to more stories and the PM could take credit for "getting so much done." This was at a certain dysfunctional, highly political networking hardware company.

nradov 8y ago
That type of dysfunction can occur under any methodology, regardless of agility. It indicates an inadequate definition of done and poor quality procedures.
aaaaaaaaaab 8y ago
So that’s why all network gear needs to be power-cycled at least weekly to function correctly!
- outworlder 8y ago
  No, although that could be part of it. You are probably talking about consumer gear. Consumers usually have two requirements: a) FEATURES FEATURES FEATURES. All of them. 200 beamforming antennas. Interplanetary internet. b) Price. All of the above for the lowest price possible. FTL communication? Cool. It better be under $50 or I'll buy the competitor. So this is what the companies optimize for. Who cares if DHCP is broken if it works most of the time. Memory leak? Just have it reboot. IPV6 broken? Most customers won't even know or care. You'll find that 'enterprise' hardware performs way better. At a price premium.
Jtsummers 8y ago
That's not agile. That's goal-driven motivation systems. "Finish twice the average stories per month and you get a bonus!" People game the rewards system by generating smaller, achievable stories. I had a colleague once who [0] would save up test cases (we were on the V&V side) to develop that were smaller. He could've had them all done and delivering value in two weeks, 200 test cases done! But if he did that he'd be left with the remaining 50 test cases that each needed 1-3 weeks to develop test systems for (sometimes shorter, I think 2 weeks or so was the average). So if he found himself looking bad on the burndown chart, he'd churn out 20 finished test cases from the easy ones. This way he was always "on track". If he'd done them and gotten ahead, they would've wondered why he slowed down (because not all test cases had the same size, duh) and penalized him for it. The problem is, that goals like that (as carrot or stick) don't change the fundamental system. If I manage to churn out 20 story points this week, but only 10 every other week, is it sustainable? Was it a fluke? Was it because I did 40 hours of OT? Or was it that they were overestimated? Maybe I had a unique skill set or insight that let me solve it more quickly than had been anticipated. Maybe something in my brain clicked (remember that moment early in your career where a "hard" concept suddenly became obvious, and all your programming tasks became much easier?) and the new level will be sustainable. On the other hand, if you create a goal to double efficiency/capacity and provide access to resources to change the development/deployment/whatever system then you can get something useful. I wrote a tool for data analysis for a project (actually a collection of related projects). The tool cut down test analysis time from hours to minutes. An activity done daily by the V&V team. I saved them years of work in that first year. The goal was to reduce the bottleneck (test results took too long to get so developers were waiting or operating blind). That goal, by itself, was worthless. It was only able to change the system of that team because I had been given permission to work on tools that would achieve it. [0] We had a manager who did things like this, though no carrot all stick.

nradov 8y ago

Agile development practices are now more relevant to the early stages of airframe design when no real hardware exists and all testing is done in computer simulations. At that point changes are relatively cheap and there are no safety concerns. But once the outer mold line is set and prototype construction starts the engineers must be far more cautious about design changes.

regularfry 8y ago

This is one of those things that sounds true, but isn't. The Saab Gripen was designed with Scrum: http://www.agilitycode.com/agile-best-practices/owning-the-sky-with-agile-saabs-jas39e-gripen-fighter-plane-building-a-jet-better-faster-cheaper-with-scrum/ http://www.agilitycode.com/agile-best-practices/owning-the-s...

SolarNet 8y ago

I mean I don't think this document is actually taking a stance on things being agile. Just on the detection of things that are actually agile and those that aren't due to the prevalence of the buzzword.

peterwwillis 8y ago

If you've ever worked in a giant industrial organization, you know it's a swamp of red tape, silos, and inefficiency. Agile is a direct response to all this. Scrum, which influenced Agile, originated from case studies of automotive, photocopier and printer industries. Agile is not 2 week sprints. Agile is not code reviews. Agile is not pair programming. Agile is abstract; it's a set of values, a philosophy, a methodology. It screams, "Stop just following some stupid formula, because it may not work for you!" The whole point is to shake up the slow, rigid, inefficient way that organizations can easily fall into, and get people to work together better. If you take the guidelines it offers and apply them, any organization can benefit. But you can't just follow a formula and be magically better. What I've learned is that all of the abstract methodologies (agile, lean, devops, etc) do not prescribe specific actions you must take; they prescribe general ways that you can adapt your work to get better outcomes. I can't give you exact evidence that it works, because there is no one way that it works. But there are tons of organizations that have followed the principles and have measurably improved their businesses. Until someone turns all of business into a science, I don't think you'll find anything better.

koonsolo 8y ago
Exactly this. The problem that people have with Agile are mostly because they think they need to follow rigid Agile rules. For me, there is only 1 true rule in Agile: good retrospectives! Once you have good retrospectives, you can improve anything within the process. Any argument about "We cannot do 2 week sprints" falls out, because your team can decide whether 2 week sprints is in your best interest.
- peterwwillis 8y ago
  Another way to refer to retrospectives would be as "Process Kaizen" (https://en.wikipedia.org/wiki/Kaizen https://en.wikipedia.org/wiki/Kaizen), or "PDCA" (https://en.wikipedia.org/wiki/PDCA https://en.wikipedia.org/wiki/PDCA). Both can be used to continuously improve pretty much anything.

wenc 8y ago

> We should probably be doing modelling and formal verification and stuff like that. This is interesting. Is this a hypothetical notion or can you share any personal experience of having worked on any projects where this is done successfully? I bring this up because there is typically a narrow band of applicability of purely theoretical constructs in most complex systems (AWS & Azure teams both use formal methods [1] to find bugs and to increase confidence in critical algorithms but I don't believe they do this system-wide. Also the verification is only done on the abstraction, not the actual implementation.). Richard Cook gave a talk on "How Complex Systems Fail" [2] which addresses this point, and advocates for designing for resiliency as opposed to designing to exhaustively handle all failure modes (which are impossible to predict). Design people often design from some idealized notion of inputs and/or worst case scenarios. Ops people know what can go wrong will go wrong, which is why ops is there to handle the deviations from the norm. Real world design exposes lift points, reveals controls and supports continuous maintenance. Highly reliable systems are evolved, not just designed. [1] https://en.wikipedia.org/wiki/TLA%2B#Industry_use https://en.wikipedia.org/wiki/TLA%2B#Industry_use [2] https://www.youtube.com/watch?v=2S0k12uZR14 https://www.youtube.com/watch?v=2S0k12uZR14

jimbokun 8y ago
"Richard Cook gave a talk on "How Complex Systems Fail" [2] which addresses this point, and advocates for designing for resiliency as opposed to designing to exhaustively handle all failure modes (which are impossible to predict)." This is the Erlang design philosophy.

jnwatson 8y ago

I'm not exactly a huge agile fan, but there are a few things it gets right. Most of the time, software development is implicitly a process of requirements development. Nobody ever knows what they want until they see it, even if they claim they do. Agile attempts to turn the implicit requirements development into explicit requirements development. It tries to shortcut the process of acquiring requirements by getting stuff in front of the user as fast as possible. Now, if you really do know the requirements (like there's a real spec to follow) or you're in a situation where you don't have the architecture decided so that you can show discrete things in a small time window (2-3 weeks), then agile isn't nearly as useful.

Jtsummers 8y ago

> 'Modern Agile' on the other hand cares about things like producing an MVP and iterating, which originated in and are appropriate in a web startup context. [emphasis added] That's a damned lie. Iterative and spiral development greatly pre-date web development. They pre-date the Web itself for software (though perhaps not the Internet, or at least Arpanet). Iterative and spiral development is the only sane way to develop any complex system (or at least to develop its specification). Agile is a further extension of that whose proponents further work to breakdown the knowledge and communication barriers that inhibit effective project development and management.

tacon 8y ago

>It's just I note this is a DOD document, and I don't think we should be designing jet fighters in 2 week sprints. Airframes probably can't be modified like that. So it follows that, within a waterfall project, the flight software shouldn't be developed agile either. There are some linked documents at the end of the OP, and this one with link "to be added"[0]. It addresses your exact point, in that hardware development and software development have to move in completely different processes. Their first example is the DoD 5000 process[1] and how it is a total fail for software: "The DoD 5000 process , depicted on the left, provides a detailed DoD process for setting requirements for complex systems and ensuring that delivered systems are compliant with those requirements. The DoD’s “one size fits all” approach to acquisition has attempted to apply this model to software systems, where it is wholly inappropriate. Software is different than hardware. Modern software methods make use of a much more iterative process, often referred to as “DevOps,” in which development and deployment (operations) are a continuous process, as depicted on the right. A key aspect of DevOps is continuous delivery of improved functionality through interaction with the end user." "DoD 5000 is designed to give OSD, the Services, and Congress some level of visibility and oversight into the development, acquisition, and sustainment of large weapons systems. While this directive may be useful for weapons systems with multi-billion dollar unit costs, it does not make sense for most software systems." [0] https://media.defense.gov/2018/Nov/02/2002058905/-1/-1/0/DIB_DOS_DONTS_%20SOFTWARE_V2_2018.11.02.PDF https://media.defense.gov/2018/Nov/02/2002058905/-1/-1/0/DIB... [1] http://acqnotes.com/wp-content/uploads/2014/09/DoD-Instruction-5000.02-The-Defense-Acquisition-System-10-Aug-17-Change-3.pdf http://acqnotes.com/wp-content/uploads/2014/09/DoD-Instructi...

myth2018 8y ago

I ask myself some similar questions. I worked for a huge Brazilian ERP company. Three years ago the board decided that the company was not realizing its full potential (which I agree with) and decided to give Agile a try. Since then, some really critical areas started to suffer a lot, the latest version is being heavily criticized by the users and the company's reputation is consistently going down. The answer seems obvious: they are doing Agile wrong, they missed the point entirely, the "big company" mindset is hindering the efforts and so on. This is, in fact, how the whole company is behaving about the poor results so far, treating any suggestion towards a Agile's rollback as a cabal proof of the company's inability in delivering software the proper way. But I dare to think that Agile might not be the only path. Most of the companies around are delivering fairly complex projects employing small and efficient and competent teams and this is really great. But some companies are not that lucky. Somebody has to build that monstrous stuff over two-decade-old legacy code foundations, employing many dozens of teams. You simply can't afford hiring the best people around, because they are not available nor willing to work for you and, anyway, there aren't enough of them. You can't work with the best users, because you have no control at all over their teams and they aren't too excited about you bothering them with your project -- and this is bad because users in Agile projects are a critical part of the team. You may somehow enforce them to work with you and this has already been made many times, but the resultant half-hearted participations are not worth the hassle. There are so many practical issues regarding a real-world agile culture implementation, so many out-of-control environmental variables affecting you very negatively, and they all seem to end up being buried under a set of principles which have allegedly worked somewhere else. As if it were easy, or even desirable, to blame your context for not being "mature enough" to adopt Agile. I've never seriously looked into it, but I believe that there are some rigorous academic case studies discussing cases where agile didn't work well. Although I believe they are scarce, both because Agile tend to fail in a smaller, yet important, set of projects, and also because they would hurt some strong beliefs which are currently being taken as common-sense. I also believe their analysis may occasionally show some bias too, perhaps focusing more on the "mistakes" made by the teams instead of considering the hypothesis of Agile being inappropriate in the analyzed case.