34 ms·

AgileFall – When Waterfall Sneaks Back into Agile

176 points by furkansahin 7y ago

softwaredoug 7y ago
Maybe for certain projects where failure is high risk waterfall is better. Or at least more up front planning is a “good thing” when fail fast = lives lost
- SamuelAdams 7y ago
  Or really complex internal projects, like an ERP / SAP implementation.
  - aeorgnoieang 7y ago
    In my mind, and agile approach would involve a series of 'tiny' incremental migrations from the existing system to the new one. I'd actually expect that to work much better than attempting to switch over everyone all at once, after years of development. That either doesn't work or works very badly in my experience.
- adrianN 7y ago
  "Fail Fast" doesn't have to mean crash airplanes. Quality and agility are orthogonal things. Agility is about understanding your requirements by producing prototypes and talking to stalkholders using those prototypes. Once you do the QA work necessary for safety critical software, misunderstandings about requirements or wrong requirements are the major source of failures.
- Ididntdothis 7y ago
  I think upfront planning is almost always a good thing. The problem starts when people think that the plan doesn’t need to be revisited. You need to be willing to change the plan as new facts develop. In the end no matter what you do software and hardware development is unpredictable. No methodology will ever change that.
  - brightball 7y ago
    This is the problem in a nutshell. Revisiting plans when you get new information. This is especially true with estimates. New work, new problems, unexpected difficulties, etc. Over-planning can lead to a lot of rigidity in the process if it tries to plan too far ahead.
    - Ididntdothis 7y ago
      “Over-planning can lead to a lot of rigidity in the process if it tries to plan too far ahead.” Or they try to compensate for technical problems by doing endless replanning meetings instead of spending the time on solving the problem. But a good long term road map is really helpful when you have to make a decision to go one way or the other. Just accept that it’s a rough guideline and not an exact plan that’s set in stone.
  - matwood 7y ago
    > I think upfront planning is almost always a good thing. Agreed. I think some people think agile means no plan, whereas I think it means frequent scheduled review of the plan as work is completed and more is learned.
  - gadders 7y ago
    Agreed. I think a lot of agile proponents think that in waterfall, the project plan is created on day 1 and as reality drifts from the plan, the project manager shouts louder and louder at developers to try get reality back to matching the plan. This a false straw man argument. One the project plan is created, it should be updated regularly (daily/weekly depending on the size of the project) as tasks are completed and new information comes to light. The plan is to give a prediction for the likely end date and cost of the project, but must be update as new information is known.
    - Ididntdothis 7y ago
      “as reality drifts from the plan, the project manager shouts louder and louder at developers to try get reality back to matching the plan” Welcome to my world :). Just a few weeks ago I had a conversation with a project manager explaining that no amount of additional planning meetings, motivational speeches or free pizza can change reality.
- pytester 7y ago
  I don't think that's true. It's better to ramp up the exposure to risk gradually once the project has proven itself in lower risk environments.
- stupidcar 7y ago
  No, it's worse. An up-front design is sometimes necessary and valuable, in both agile or waterfall projects, but it always carries a risk of including mistakes or mistaken assumptions that result in dangerous flaws. When such a flaw is discovered, especially late in the project, it is often very expensive and difficult to fix, because it has become foundational to the solution. This is true regardless of whether an waterfall or agile process has been adopted, and there will often be significant pressure to implement a hacky fix, instead of the truly correct one. Where agile differs from waterfall is that it does, at least, philosophically accept that nothing is set in stone, even at a late stage. Discussions regarding how to address a flaw can then proceed on a basis of cost vs. risk vs. difficulty, and a hopefully rational choice can be made. In contrast, the rigidity of the waterfall process is frequently used as a bureaucratic weapon to prevent these discussions from even happening. Decisions made in the project's initial stages are held to be settled and unalterable, and those seeking to overturn them usually have to fight a horrendous, uphill battle to do so, even when they're in the right. I would liken the difference between design in agile and waterfall to that of scientific theory vs religious dogma. Neither necessarily has a monopoly on truth: A theory might be mistaken while dogma, by chance, might be correct. And challenging well-established scientific theory can, in practice, be almost as difficult and painful as disputing dogma. But science at least accepts the possibility of change, however hard it may be.
  - fernandopj 7y ago
    > In contrast, the rigidity of the waterfall process is frequently used as a bureaucratic weapon to prevent these discussions from even happening. Decisions made in the project's initial stages are held to be settled and unalterable, and those seeking to overturn them usually have to fight a horrendous, uphill battle to do so, even when they're in the right. So much this. I can't count how many times I had a client tell me "We want the delivery on the proposed time, on budget, I don't care" even when analysing the client's design it shows obvious flaws and won't work as intended. Most of them usually gets fired from doing this, months later, but even so they just get another same-level job on another company, and the cycle goes on. There's no development methology that will ever work on irrational business demands, and that happens all the time. Sometimes I think HN's users are just too entrenched at product-driven companies, or B2C ones, and never get to experience the amount of BS and CYA that happens on B2B, large-business deals.
SamuelAdams 7y ago
Based on this: >Teams only checked in – wait for it – every three months in a formal schedule review. I listened as Henrich mentioned that the teams complained about the volume of paperwork he makes them fill out for these quarterly reviews. In my experience, embracing agile is an organizational change, not just an IT team change. Product owners and subject matter experts need to be more engaged with their development teams. Not all business folks want to do this: they have their own day-jobs to get to. So when the IT team starts iterating faster, but the appropriate business / project stakeholders do not change their workflow, you get this "AgileFall" combo that just does not really work.
- Ididntdothis 7y ago
  That’s exactly how agile got introduced at my company. The scrum training was attended by most devs, only a few higher ranked architects, exactly one level 1 manager. Nobody from marketing, project management or upper management attended and now years later they still don’t understand how Agile is supposed to work. For them it’s something engineers do. So in the end nothing really changed other than words.
gnud 7y ago
The last full system I delivered was created using a sort of waterfall/agile hybrid. We worked from a backlog, in short-ish sprints, but nothing was put into production until a certain milestone was reached, after multiple sprints. We were replacing an existing, legacy system, and even though we switched over "piece by piece", each module of the old system was at least 10-12 sprints. And the end-users couldn't do a meaningful test of their normal tasks until most of each module was implemented. How to teams who manage to work more "agile" do this sort of thing?
- dtech 7y ago
  This seams a pretty reasonable approach to solve this problem. The only improvement I could think of is to "mirror" the data or API calls to the new system so it gets battle-tested earlier, but that only works for backend systems.
  - gnud 7y ago
    We did that for parts of the system, but that mirroring also needed to be implemented and tested. So most modules we straight out replaced once the new versions were done.
  - beat 7y ago
    That only works with similar architectures. I've been involved with converting mainframe batch systems to service-oriented architectures on distributed systems, and that sort of mirroring does not work at all. It's fine for prying microservices loose from monoliths for relatively modern REST api systems, tho. Big problems I see there are that the problem often isn't the code, but rather the api itself that's a mess, and evolving the api breaks client code which you may or may not control.
    - dtech 7y ago
      > converting mainframe batch systems [...], and that sort of mirroring does not work at all. Interesting, what are the problems you encounter if you try that?
roland35 7y ago
AgileFall does sound like an unholy mess, taking the worst aspects of both Agile and Waterfall! However, I still think waterfall is the only approach when trying to design a hardware product. Electronics require pretty specific requirements up front before you design and order boards, obviously mechanical tooling can be even more expensive. But... I think there are lots of ways to do quick experiments and tighten up the feedback look even with robotics or electronics. Breadboards and development kits can be used as initial electronics prototypes. 3d prints help test a mechanical concept. Within each phase of the design more experiments can be done. But ultimately the project is probably more waterfall than agile!
- supahfly_remix 7y ago
  I've done both hardware and software projects. My hardware projects began with an analysis phase where feasibility studies were done to answer questions like, can we build it at the desired frequency with current technology? No actual work on the project would be done at this point, since it was at risk of being thrown away. Modern software projects seem to just start with people typing and little upfront documentation. A long time ago there was documentation (flow charts, Booch diagrams). Now whether these are useful seems debatable, but I do not see any upfront analysis done with software.
  - mikekchar 7y ago
    With hardware you can check your designs against the laws of physics. With software there usually aren't any such laws. Being a guy who has actually written a 1300 page requirements document in my career as well as large books full of designs, my experience has been that the only thing you can test them against is code. Usually you are wrong :-). It's much better to start off with tiny pieces and make sure to write a lot of code along the way. Exactly how much upfront analysis you need before you start writing code is a judgement call, but I would say that it's almost always safer to err on the side of writing code too soon.
- lmm 7y ago
  Agile exists for the particular constraints of software development, where requirements are unknown, change is relatively cheap, and feedback loops are very tight. I wouldn't expect it to be appropriate to other domains.
netfl0 7y ago
This article conflates the reporting and engineering aspects of the two processes, then it goes on to focus purely on the reporting. There was also no discussion of engineering discipline and how that fits into their process, despite the waterfall graphics showing requirements and design.
Spearchucker 7y ago
None of it even matters. Both agile and waterfall can succeed. Mostly in spite of the chosen methodology, not because of it.
- crimsonalucard 7y ago
  Agile is often more bureaucratic overhead. More meetings, more checking in.
  - AnimalMuppet 7y ago
    At the risk of "no true Scotsman"... If it's creating more overhead, it's not agile. It may wear the label, but it's not the real thing. If the existing process was already the optimal one, the agile thing to do is to not change it. If the "agile" changes are adding overhead, the actually agile thing to do is to go back to the previous process.
    - crimsonalucard 7y ago
      So basically the definition of agile is: "The perfect way to do project management" With this definition in place the future is solid, no criticism can bring agile down because agile can evolve into whatever you want it to be. Literally, what is the difference between "agile" and "whatever works"?
      - AnimalMuppet 7y ago
        Agile is "adjust your process". Your process is one of the variables, not something handed down from on high. If you aren't adjusting your process, you aren't doing agile. Many places say "We will be agile according to this rigidly defined process". The contradiction is obvious.
        crimsonalucard 7y ago
        Whatever works man, whatever works.
pytester 7y ago
IME the most common cause for waterfall sneaking back into agile is when agile experiences some kind of failure - like the project delivered late and over budget. The response to this is often to revert to waterfall methods and means. This frequently makes things worse. There's a common misperception in the industry that agile is 'fast' but waterfall is better at handling risky situations.
- NikolaNovak 7y ago
  >> [perception that] waterfall is better at handling risky situations. Subtle difference I think: Waterfall is perceived as being more predictable, which is why it would seem desirable when stability is needed. Fully agreed that while Agile is meant to deal with change better, it's perceived as being a silver bullet to "Speed things up". I have yet to meet a enterprise manager who uses the word correctly :(
  - beat 7y ago
    I once wrote a blog article ranting about how we shouldn't use the word "sprint", because it focuses on speed. Rather, we should say "iteration", which focuses on change. Pushing agile as faster rather than better is the biggest problem agile ever had.
    - NikolaNovak 7y ago
      I absolutely positively could not agree more. Majority of my managers have heard the word somewhere, tried to infer its meaning from dictionary definition, and started using it arbitrarily. Most frequent phrase is "We need to be more AGILE!", which 98.72% of the time simply means "We need to whip the workers more and have them work harder, faster, longer.... somehow".
- zaphar 7y ago
  That's ironic, because in many cases the reverse is true. Agile can be slower when you are exploring the solution space. It's safer though in that scenario since you are more likely to identify and fix issues as you enter them.
ausjke 7y ago
Been in agile team for about a year now, the only good thing I feel is the stand-up meetings, that reminds everyone to stay on track, and update the status each morning(also press you ahead, can not be too lazy due to this daily update thing, especially for old lazy engineers like me), also, a quick exchange of ideas in the standup can make a day more productive once a while. This is the major if not the only thing I'm going to keep for the future.
- vbtemp 7y ago
  Fascinating. The worst thing about agile I feel is the stand-up meeting. It encourages people just to say stuff and posture as being busy. I feel it encourages the worst part about agile -- that input (work) is more important than results (a working product).
  - himynameisdom 7y ago
    If the team isn't relaying how their work is progressing towards the sprint goal, the daily scrum ends up turning into a status meaning, negating its intended purpose. The daily scrum is a planning meeting. It's meant to help the team understand the big picture scoped to 1 sprint while helping each other achieve a shared commitment.
  - smileysteve 7y ago
    I've been on a variety of teams that do this differently. I once blew up when multiple people said that they were in [the scrum] meetings the day before; this is an example of saying things to account for your time, not to ensure the team is on the same page! I personally try to correct from this problem by asking questions whenever work may relate to me, and being very familiar with the backlog to know how things will coordinate. But also, I rant when people tell me about meetings without any conclusions for the team.
    - dane-pgp 7y ago
      > I once blew up when multiple people said that they were in [the scrum] meetings the day before; this is an example of saying things to account for your time Have you considered the possibility that your temperament or personality might be the reason that multiple people on your team feel they have to pre-emptively justify themselves to avoid angry criticism? A meeting where people have to stand in front of their peers and potentially admit their failings can be quite anxiety-inducing if team members can't be trusted to respond in a professional manner.
dragonwriter 7y ago
“Agile” and “Waterfall” aren't opposed, or even necessarily incompatible; “Agile” is opposed to “Process not dynamically adapted to the needs of the particular team in developing working software”. I'd bet that these days it's a lot more common for a process imposed in the way Agile is a reaction against to be some variant of Scrum with which management is enamored, rather than waterfall.
crimsonalucard 7y ago
Buzzwords like agile and agilefall and waterfall don't really mean anything. One persons anecdotal experience about agilefall doesn't mean it doesn't work. Don't fall for the trap of using analogies and anecdotal experiences as evidence. To top it off this guy didn't even measure the outcome of agilefall, all he did was declare that it violates some made up philosophical principles and demand change. I've seen a project manager literally catch himself from doing something waterfall like it was the plague. Do people not realize that waterfall is the only technique used to build airplanes and bridges? I would not get on a plane built/being built using agile. Note that I'm not criticizing agile. Agile works, waterfall works and no logic or evidence provided by that article says that combining what works for both is not good.
- commandlinefan 7y ago
  > agilefall and waterfall don't really mean anything That's the problem, though - they should. (And, actually, they do, but they've been twisted around and used incorrectly so often that they've come to mean nothing).
- aeorgnoieang 7y ago
  Of course "anecdotal experiences" are evidence. Yes, they're messy, and may not be applicable to other circumstances, but they're certainly informative. I'm pretty sure airplanes and bridges aren't designed using 'waterfall'. I'm sure they're built, i.e. 'deployed', using a well-defined process. But, ideally, the well-defined process by which they're built should itself probably be designed or developed with 'short' iterations.
  - crimsonalucard 7y ago
    >Of course "anecdotal experiences" are evidence. You're right. It is evidence. This is an obvious fact that I mis-represented on accident. Let me rephrase for clarity: "This persons anecdotal evidence is not good enough or unbiased enough to qualify as evidence to be used to arrive at a conclusion" >I'm pretty sure airplanes and bridges aren't designed using 'waterfall'. I'm sure they're built, i.e. 'deployed', using a well-defined process. Ok let me rephrase for more clarity in case people don't have the ability to know that by "designing" a bridge I can mean "build" and "deploy" as well. "Airplanes and bridges are designed and built using waterfall." >But, ideally, the well-defined process by which they're built should itself probably be designed or developed with 'short' iterations. This was how the MAX part of the 737 MAX was designed. Using a short iteration on top of an existing design. It leads to shorter turn around and cheaper costs. With the small expense of 1 or 2 plane crashes. If you can afford a plane crash go agile, if you want to build something correctly go waterfall.
peterwwillis 7y ago
At Cisco there was a big organizational push to become "an Agile organization". What we ended up with was waterfall with Scrum. But regardless of people's complaints, and the lack of integration throughout the org, we still got products out the door when we said we would, worked on solutions iteratively, added value, etc. If all you focus on is the name of your process, rather than just getting work done, you're not gonna get far. I find a lack of a vision, strategy, roadmap, etc to be way worse than just doing something in a less-than-Agile way.
- himynameisdom 7y ago
  Agreed. Vision and strategy set the table for why and how the work moves towards an intended outcome. Every company I've worked with that does scrumfall tends to value outputs over outcomes of the outputs. The process name doesn't matter, the results of the process should be the focus of every product company.
patsplat 7y ago
Waterfall isn't an engineering methodology, it's an investment methodology. Engineering will mirror the interests of the capital. And sometimes capital wants to build the same thing again for a "safe" return, rather than building something new for a transformative return. IMHO it's not actually a safe investment strategy, but I understand where it comes from.
deleted 7y ago
[deleted]
megiddo 7y ago
Waterscrumban is my favorite method.
- beat 7y ago
  Yeah, we did Waterscrumban. We're in our third sprint. The first sprint, we moved "requirements" from backlog to doing. Second spring, we moved "requirements" to done, and moved "design" to doing. Third sprint, we moved "design" to done, and "implementation" to doing. Verification and maintenance are still in the backlog. It works great! Just three more iterations, and we'll be done with maintenance!
forgottenpass 7y ago
Here's an example of why I still can't take writing about Agile seriously. I'm not saying there aren't a lot of process improvements under the banner of Agile (and plenty more that Agile is retroactively claiming credit for), but writing like this turns me off: >Henrich still had a process where success was measured by reports, not outcomes. It was the same reporting process used to measure projects that used linear, step-by-step Waterfall. What are reports, if not documentation of outcomes? What are outcomes without evidence that those outcomes happened? I can see what he's saying, but the more accurate description of the problem "the reports included unnecessary and distracting information." But that doesn't make as good of a blog post as casting it as "paperwork vs understanding outcomes." The whole thing feels like it needs to invent a "paperwork = evil" boogieman just to allow a process change that throws away useless paperwork and allow a team to focus on the paperwork that you just don't call paperwork anymore. Email is paperwork. Version control is paperwork. Change management is paperwork. It doesn't matter if it's software updating a database instead of a secretary updating 3-ring binders. Most of development is paperwork. In fact, online databases mean we do way more paperwork. If we were bottlenecked by that sea of file cabinets in the basement containing all our the legacy projects, we would have way less paperwork just out of necessity. Standups are paperwork that just happen to not use paper.
- igouy 7y ago
  > Standups are paperwork that just happen to not use paper. If it isn't written down, it didn't happen! (Memory is not an infallible record.)
rezeroed 7y ago
"Sneaks"? There are projects where the agile coaches, scrum masters, and consultants proudly declare regularly that the project embraces the best of both agile and waterfall. Nothing resembling sneakiness.
baxtr 7y ago
AgileFall is coming!
rhombocombus 7y ago
I am currently leading a dev team on a project that I would describe as waterfagile/agilefall. It's a highly regulated industry and the product is for a state level client that has rigorous specifications. We operate iteratively, but there is a discrete golive and many specifications of the project were defined contractually before we pushed a single line of code. For this kind of project it doesn't make sense to operate in a pure agile fashion, but we also have broad license to operate how we please in some aspects of the project (i.e. the UI), and those more closely resemble an agile method project.
jcims 7y ago
We're struggling, IMHO, with adopting agile in a highly regulated environment where there are a multitude of tollgates to get to production. Roughly 20 different stakeholders that need to be accommodated while leveraging many internal services that do not support automation. Slipping back into waterfall can happen because many of these issues have sequential dependencies and are one time request driven processes. It's a bit of a nightmare.
kelnos 7y ago
I've heard -- and much prefer -- the term "Waterfragile". It's funnier, and IMO more descriptive of what's going on.
- NateEag 7y ago
  Agreed. I was quite disappointed the day it came to me, I searched for it, and discovered I was not the coiner.
eagsalazar2 7y ago
Obviously not every detail is included here but I did find it odd that, although he mentions a focus on outcomes, the change he talks about is entirely about shifting from one process to another, both of which by themselves might be pretty arbitrary. > reviews would be about frequent delivery, incremental development and how leadership could remove obstacle I would think that in a Lean process that outcomes and therefore reports would be focused on performance of North Star Metrics, KPIs, comparing those against goals, analytics/research to help understand that performance, and hypotheses/strategies for how to improve. I'm a big fan of Lean/BML approach and I'm assuming this is what is happening but the article doesn't really send that message. Because of that omission, after reading but before seeing the author my first thought was this was consultant blather.
trentnix 7y ago
It seems every time I read an article where some Agile consultant extols the evils of waterfall the author commits a Just In Case Fallacy or the lede is buried. This article is no different: Teams only checked in – wait for it – every three months in a formal schedule review. I listened as Henrich mentioned that the teams complained about the volume of paperwork he makes them fill out for these quarterly reviews. And he was unhappy with the quality of the reports because he felt most teams wrote the reports the night before the review. That's not a problem with AgileFall or ScrumFall, that's ineffective management irrespective of how the sausage is made. But nobody has to wear the Scarlet Letter of Project Failure if we assign blame to the process! I've also never found that pure (like the wind-driven snow capital-A) Agile matches reality. After spending more than a decade wrestling with it - I still haven't figured out how quality assurance and test fits into the same iteration in Agile. Most teams I inherit have non-technical QA resources. Most projects I inherit have complex business logic and integration needs that aren't easily testable until some state of development "done" has been achieved. And time and time again I find myself staggering their effort slightly behind development so that test and development resources aren't taking turns sitting on their asses waiting for the other side. I've written about this before as was even been publicly admonished by an Agile Manifesto signatory (hallowed be their names) that I was doing it WRONG WRONG WRONG! But I was never able to figure out how to contort reality to satisfy their criticism. Believing in iterations and backlogs and prioritization and shortened feedback loops wasn't enough. Thinking QA - any QA! - occurred AFTER any development was done was altogether impure. And despite asking repeatedly for concrete examples of how its done the "right" way, the only responses I received were vague platitudes and consultant sales blurbs that I guess I wasn't smart enough to understand. In the end, I don't think there's any escaping that work has to be done in a sequence: identify a need, identify a solution, implement the solution, validate the solution. And so every software project has at least a little bit of waterfall mixed in.
- Groxx 7y ago
  My Grand Unifying Theory of Explaining Agile Proselytizers™ is "if it's not working, you're doing it wrong". True Agile always works. Oh, and it has more-frequent check-ins/iterations than Waterfall (which is the defining aspect of Waterfall: fewer check-ins/iterations than Agile). The actual frequency is undefined, it exists only in comparison to Waterfall. --- The problem is that that kind of stance does nothing but fluff up agile consultant's resumes. It doesn't actually lead to improvements that last, or even discussion that might lead to improvements. Agile is perfect because it's Agile.
bitwize 7y ago
I've never seen an Agile implementation in the wild that wasn't a game of Mornington Crescent, whose stated goal was closer adherence to Agile principles but whose actual goal was to reinforce existing power structures and undermine developer autonomy.
codeulike 7y ago
When feeling troublesome I like to endorse Agile enthusiasts with the skill 'Waterfall' on LinkedIn
api 7y ago
I hate when buzzword sneaks into buzzword.
LifeLiverTransp 7y ago
Power corrupts all methods. Power needs (informal) hierarchy, hierarchy needs communication and coordination limitations. In agile, the customer, usually a arcane diety, shrouded away by requirement engineers and the management, is actually part of the team, and lets face it, the ultimate subversion to all management power. If you can convince the customer representive, that this or that feature is the most important, all the management talk is in the icebox. Which can bring out all the ugly parts in agile. No longterm thinking, just shorterm fast movement, with architectue problems later in the game. So management even has a reason, to persist on "communication" channels. Which is - when agile comes to die in long meetings, made filled with warped perception bubbles. Use Contract First Design and try something wild - a approach that allows your paradigm to pulse between nearly agile and "waterfally" repair-cycles to the damage done to the longterm goals. And be aware that there might be diverging interests- of which none are evil.
bastijn 7y ago
Coming from the other side we have Scaled Agile Framework (SAFe) [0]. Which introduces some waterfall to Agile instead of the other way around. 3 month planning events, commitments, etc. Not bad perse if you work in an environment where not everybody is agile or every process can be agile. [0] https://www.scaledagileframework.com/ https://www.scaledagileframework.com/
effnorwood 7y ago
*AgileFail

176 points by furkansahin 7y ago

Maybe for certain projects where failure is high risk waterfall is better. Or at least more up front planning is a “good thing” when fail fast = lives lost

SamuelAdams 7y ago
Or really complex internal projects, like an ERP / SAP implementation.
- aeorgnoieang 7y ago
  In my mind, and agile approach would involve a series of 'tiny' incremental migrations from the existing system to the new one. I'd actually expect that to work much better than attempting to switch over everyone all at once, after years of development. That either doesn't work or works very badly in my experience.
adrianN 7y ago
"Fail Fast" doesn't have to mean crash airplanes. Quality and agility are orthogonal things. Agility is about understanding your requirements by producing prototypes and talking to stalkholders using those prototypes. Once you do the QA work necessary for safety critical software, misunderstandings about requirements or wrong requirements are the major source of failures.
Ididntdothis 7y ago
I think upfront planning is almost always a good thing. The problem starts when people think that the plan doesn’t need to be revisited. You need to be willing to change the plan as new facts develop. In the end no matter what you do software and hardware development is unpredictable. No methodology will ever change that.
- brightball 7y ago
  This is the problem in a nutshell. Revisiting plans when you get new information. This is especially true with estimates. New work, new problems, unexpected difficulties, etc. Over-planning can lead to a lot of rigidity in the process if it tries to plan too far ahead.
  - Ididntdothis 7y ago
    “Over-planning can lead to a lot of rigidity in the process if it tries to plan too far ahead.” Or they try to compensate for technical problems by doing endless replanning meetings instead of spending the time on solving the problem. But a good long term road map is really helpful when you have to make a decision to go one way or the other. Just accept that it’s a rough guideline and not an exact plan that’s set in stone.
- matwood 7y ago
  > I think upfront planning is almost always a good thing. Agreed. I think some people think agile means no plan, whereas I think it means frequent scheduled review of the plan as work is completed and more is learned.
- gadders 7y ago
  Agreed. I think a lot of agile proponents think that in waterfall, the project plan is created on day 1 and as reality drifts from the plan, the project manager shouts louder and louder at developers to try get reality back to matching the plan. This a false straw man argument. One the project plan is created, it should be updated regularly (daily/weekly depending on the size of the project) as tasks are completed and new information comes to light. The plan is to give a prediction for the likely end date and cost of the project, but must be update as new information is known.
  - Ididntdothis 7y ago
    “as reality drifts from the plan, the project manager shouts louder and louder at developers to try get reality back to matching the plan” Welcome to my world :). Just a few weeks ago I had a conversation with a project manager explaining that no amount of additional planning meetings, motivational speeches or free pizza can change reality.
pytester 7y ago
I don't think that's true. It's better to ramp up the exposure to risk gradually once the project has proven itself in lower risk environments.
stupidcar 7y ago
No, it's worse. An up-front design is sometimes necessary and valuable, in both agile or waterfall projects, but it always carries a risk of including mistakes or mistaken assumptions that result in dangerous flaws. When such a flaw is discovered, especially late in the project, it is often very expensive and difficult to fix, because it has become foundational to the solution. This is true regardless of whether an waterfall or agile process has been adopted, and there will often be significant pressure to implement a hacky fix, instead of the truly correct one. Where agile differs from waterfall is that it does, at least, philosophically accept that nothing is set in stone, even at a late stage. Discussions regarding how to address a flaw can then proceed on a basis of cost vs. risk vs. difficulty, and a hopefully rational choice can be made. In contrast, the rigidity of the waterfall process is frequently used as a bureaucratic weapon to prevent these discussions from even happening. Decisions made in the project's initial stages are held to be settled and unalterable, and those seeking to overturn them usually have to fight a horrendous, uphill battle to do so, even when they're in the right. I would liken the difference between design in agile and waterfall to that of scientific theory vs religious dogma. Neither necessarily has a monopoly on truth: A theory might be mistaken while dogma, by chance, might be correct. And challenging well-established scientific theory can, in practice, be almost as difficult and painful as disputing dogma. But science at least accepts the possibility of change, however hard it may be.
- fernandopj 7y ago
  > In contrast, the rigidity of the waterfall process is frequently used as a bureaucratic weapon to prevent these discussions from even happening. Decisions made in the project's initial stages are held to be settled and unalterable, and those seeking to overturn them usually have to fight a horrendous, uphill battle to do so, even when they're in the right. So much this. I can't count how many times I had a client tell me "We want the delivery on the proposed time, on budget, I don't care" even when analysing the client's design it shows obvious flaws and won't work as intended. Most of them usually gets fired from doing this, months later, but even so they just get another same-level job on another company, and the cycle goes on. There's no development methology that will ever work on irrational business demands, and that happens all the time. Sometimes I think HN's users are just too entrenched at product-driven companies, or B2C ones, and never get to experience the amount of BS and CYA that happens on B2B, large-business deals.

SamuelAdams 7y ago

Based on this: >Teams only checked in – wait for it – every three months in a formal schedule review. I listened as Henrich mentioned that the teams complained about the volume of paperwork he makes them fill out for these quarterly reviews. In my experience, embracing agile is an organizational change, not just an IT team change. Product owners and subject matter experts need to be more engaged with their development teams. Not all business folks want to do this: they have their own day-jobs to get to. So when the IT team starts iterating faster, but the appropriate business / project stakeholders do not change their workflow, you get this "AgileFall" combo that just does not really work.

Ididntdothis 7y ago
That’s exactly how agile got introduced at my company. The scrum training was attended by most devs, only a few higher ranked architects, exactly one level 1 manager. Nobody from marketing, project management or upper management attended and now years later they still don’t understand how Agile is supposed to work. For them it’s something engineers do. So in the end nothing really changed other than words.

gnud 7y ago

The last full system I delivered was created using a sort of waterfall/agile hybrid. We worked from a backlog, in short-ish sprints, but nothing was put into production until a certain milestone was reached, after multiple sprints. We were replacing an existing, legacy system, and even though we switched over "piece by piece", each module of the old system was at least 10-12 sprints. And the end-users couldn't do a meaningful test of their normal tasks until most of each module was implemented. How to teams who manage to work more "agile" do this sort of thing?

dtech 7y ago
This seams a pretty reasonable approach to solve this problem. The only improvement I could think of is to "mirror" the data or API calls to the new system so it gets battle-tested earlier, but that only works for backend systems.
- gnud 7y ago
  We did that for parts of the system, but that mirroring also needed to be implemented and tested. So most modules we straight out replaced once the new versions were done.
- beat 7y ago
  That only works with similar architectures. I've been involved with converting mainframe batch systems to service-oriented architectures on distributed systems, and that sort of mirroring does not work at all. It's fine for prying microservices loose from monoliths for relatively modern REST api systems, tho. Big problems I see there are that the problem often isn't the code, but rather the api itself that's a mess, and evolving the api breaks client code which you may or may not control.
  - dtech 7y ago
    > converting mainframe batch systems [...], and that sort of mirroring does not work at all. Interesting, what are the problems you encounter if you try that?

roland35 7y ago

AgileFall does sound like an unholy mess, taking the worst aspects of both Agile and Waterfall! However, I still think waterfall is the only approach when trying to design a hardware product. Electronics require pretty specific requirements up front before you design and order boards, obviously mechanical tooling can be even more expensive. But... I think there are lots of ways to do quick experiments and tighten up the feedback look even with robotics or electronics. Breadboards and development kits can be used as initial electronics prototypes. 3d prints help test a mechanical concept. Within each phase of the design more experiments can be done. But ultimately the project is probably more waterfall than agile!

supahfly_remix 7y ago
I've done both hardware and software projects. My hardware projects began with an analysis phase where feasibility studies were done to answer questions like, can we build it at the desired frequency with current technology? No actual work on the project would be done at this point, since it was at risk of being thrown away. Modern software projects seem to just start with people typing and little upfront documentation. A long time ago there was documentation (flow charts, Booch diagrams). Now whether these are useful seems debatable, but I do not see any upfront analysis done with software.
- mikekchar 7y ago
  With hardware you can check your designs against the laws of physics. With software there usually aren't any such laws. Being a guy who has actually written a 1300 page requirements document in my career as well as large books full of designs, my experience has been that the only thing you can test them against is code. Usually you are wrong :-). It's much better to start off with tiny pieces and make sure to write a lot of code along the way. Exactly how much upfront analysis you need before you start writing code is a judgement call, but I would say that it's almost always safer to err on the side of writing code too soon.
lmm 7y ago
Agile exists for the particular constraints of software development, where requirements are unknown, change is relatively cheap, and feedback loops are very tight. I wouldn't expect it to be appropriate to other domains.

netfl0 7y ago

This article conflates the reporting and engineering aspects of the two processes, then it goes on to focus purely on the reporting. There was also no discussion of engineering discipline and how that fits into their process, despite the waterfall graphics showing requirements and design.

Spearchucker 7y ago

None of it even matters. Both agile and waterfall can succeed. Mostly in spite of the chosen methodology, not because of it.

crimsonalucard 7y ago
Agile is often more bureaucratic overhead. More meetings, more checking in.
- AnimalMuppet 7y ago
  At the risk of "no true Scotsman"... If it's creating more overhead, it's not agile. It may wear the label, but it's not the real thing. If the existing process was already the optimal one, the agile thing to do is to not change it. If the "agile" changes are adding overhead, the actually agile thing to do is to go back to the previous process.
  - crimsonalucard 7y ago
    So basically the definition of agile is: "The perfect way to do project management" With this definition in place the future is solid, no criticism can bring agile down because agile can evolve into whatever you want it to be. Literally, what is the difference between "agile" and "whatever works"?
    - AnimalMuppet 7y ago
      Agile is "adjust your process". Your process is one of the variables, not something handed down from on high. If you aren't adjusting your process, you aren't doing agile. Many places say "We will be agile according to this rigidly defined process". The contradiction is obvious.
      - crimsonalucard 7y ago
        Whatever works man, whatever works.

pytester 7y ago

IME the most common cause for waterfall sneaking back into agile is when agile experiences some kind of failure - like the project delivered late and over budget. The response to this is often to revert to waterfall methods and means. This frequently makes things worse. There's a common misperception in the industry that agile is 'fast' but waterfall is better at handling risky situations.

NikolaNovak 7y ago
>> [perception that] waterfall is better at handling risky situations. Subtle difference I think: Waterfall is perceived as being more predictable, which is why it would seem desirable when stability is needed. Fully agreed that while Agile is meant to deal with change better, it's perceived as being a silver bullet to "Speed things up". I have yet to meet a enterprise manager who uses the word correctly :(
- beat 7y ago
  I once wrote a blog article ranting about how we shouldn't use the word "sprint", because it focuses on speed. Rather, we should say "iteration", which focuses on change. Pushing agile as faster rather than better is the biggest problem agile ever had.
  - NikolaNovak 7y ago
    I absolutely positively could not agree more. Majority of my managers have heard the word somewhere, tried to infer its meaning from dictionary definition, and started using it arbitrarily. Most frequent phrase is "We need to be more AGILE!", which 98.72% of the time simply means "We need to whip the workers more and have them work harder, faster, longer.... somehow".
zaphar 7y ago
That's ironic, because in many cases the reverse is true. Agile can be slower when you are exploring the solution space. It's safer though in that scenario since you are more likely to identify and fix issues as you enter them.

ausjke 7y ago

Been in agile team for about a year now, the only good thing I feel is the stand-up meetings, that reminds everyone to stay on track, and update the status each morning(also press you ahead, can not be too lazy due to this daily update thing, especially for old lazy engineers like me), also, a quick exchange of ideas in the standup can make a day more productive once a while. This is the major if not the only thing I'm going to keep for the future.

vbtemp 7y ago
Fascinating. The worst thing about agile I feel is the stand-up meeting. It encourages people just to say stuff and posture as being busy. I feel it encourages the worst part about agile -- that input (work) is more important than results (a working product).
- himynameisdom 7y ago
  If the team isn't relaying how their work is progressing towards the sprint goal, the daily scrum ends up turning into a status meaning, negating its intended purpose. The daily scrum is a planning meeting. It's meant to help the team understand the big picture scoped to 1 sprint while helping each other achieve a shared commitment.
- smileysteve 7y ago
  I've been on a variety of teams that do this differently. I once blew up when multiple people said that they were in [the scrum] meetings the day before; this is an example of saying things to account for your time, not to ensure the team is on the same page! I personally try to correct from this problem by asking questions whenever work may relate to me, and being very familiar with the backlog to know how things will coordinate. But also, I rant when people tell me about meetings without any conclusions for the team.
  - dane-pgp 7y ago
    > I once blew up when multiple people said that they were in [the scrum] meetings the day before; this is an example of saying things to account for your time Have you considered the possibility that your temperament or personality might be the reason that multiple people on your team feel they have to pre-emptively justify themselves to avoid angry criticism? A meeting where people have to stand in front of their peers and potentially admit their failings can be quite anxiety-inducing if team members can't be trusted to respond in a professional manner.

dragonwriter 7y ago

“Agile” and “Waterfall” aren't opposed, or even necessarily incompatible; “Agile” is opposed to “Process not dynamically adapted to the needs of the particular team in developing working software”. I'd bet that these days it's a lot more common for a process imposed in the way Agile is a reaction against to be some variant of Scrum with which management is enamored, rather than waterfall.

crimsonalucard 7y ago

Buzzwords like agile and agilefall and waterfall don't really mean anything. One persons anecdotal experience about agilefall doesn't mean it doesn't work. Don't fall for the trap of using analogies and anecdotal experiences as evidence. To top it off this guy didn't even measure the outcome of agilefall, all he did was declare that it violates some made up philosophical principles and demand change. I've seen a project manager literally catch himself from doing something waterfall like it was the plague. Do people not realize that waterfall is the only technique used to build airplanes and bridges? I would not get on a plane built/being built using agile. Note that I'm not criticizing agile. Agile works, waterfall works and no logic or evidence provided by that article says that combining what works for both is not good.

commandlinefan 7y ago
> agilefall and waterfall don't really mean anything That's the problem, though - they should. (And, actually, they do, but they've been twisted around and used incorrectly so often that they've come to mean nothing).
aeorgnoieang 7y ago
Of course "anecdotal experiences" are evidence. Yes, they're messy, and may not be applicable to other circumstances, but they're certainly informative. I'm pretty sure airplanes and bridges aren't designed using 'waterfall'. I'm sure they're built, i.e. 'deployed', using a well-defined process. But, ideally, the well-defined process by which they're built should itself probably be designed or developed with 'short' iterations.
- crimsonalucard 7y ago
  >Of course "anecdotal experiences" are evidence. You're right. It is evidence. This is an obvious fact that I mis-represented on accident. Let me rephrase for clarity: "This persons anecdotal evidence is not good enough or unbiased enough to qualify as evidence to be used to arrive at a conclusion" >I'm pretty sure airplanes and bridges aren't designed using 'waterfall'. I'm sure they're built, i.e. 'deployed', using a well-defined process. Ok let me rephrase for more clarity in case people don't have the ability to know that by "designing" a bridge I can mean "build" and "deploy" as well. "Airplanes and bridges are designed and built using waterfall." >But, ideally, the well-defined process by which they're built should itself probably be designed or developed with 'short' iterations. This was how the MAX part of the 737 MAX was designed. Using a short iteration on top of an existing design. It leads to shorter turn around and cheaper costs. With the small expense of 1 or 2 plane crashes. If you can afford a plane crash go agile, if you want to build something correctly go waterfall.

peterwwillis 7y ago

At Cisco there was a big organizational push to become "an Agile organization". What we ended up with was waterfall with Scrum. But regardless of people's complaints, and the lack of integration throughout the org, we still got products out the door when we said we would, worked on solutions iteratively, added value, etc. If all you focus on is the name of your process, rather than just getting work done, you're not gonna get far. I find a lack of a vision, strategy, roadmap, etc to be way worse than just doing something in a less-than-Agile way.

himynameisdom 7y ago
Agreed. Vision and strategy set the table for why and how the work moves towards an intended outcome. Every company I've worked with that does scrumfall tends to value outputs over outcomes of the outputs. The process name doesn't matter, the results of the process should be the focus of every product company.

patsplat 7y ago

Waterfall isn't an engineering methodology, it's an investment methodology. Engineering will mirror the interests of the capital. And sometimes capital wants to build the same thing again for a "safe" return, rather than building something new for a transformative return. IMHO it's not actually a safe investment strategy, but I understand where it comes from.

deleted 7y ago

[deleted]

megiddo 7y ago

Waterscrumban is my favorite method.

beat 7y ago
Yeah, we did Waterscrumban. We're in our third sprint. The first sprint, we moved "requirements" from backlog to doing. Second spring, we moved "requirements" to done, and moved "design" to doing. Third sprint, we moved "design" to done, and "implementation" to doing. Verification and maintenance are still in the backlog. It works great! Just three more iterations, and we'll be done with maintenance!

forgottenpass 7y ago

Here's an example of why I still can't take writing about Agile seriously. I'm not saying there aren't a lot of process improvements under the banner of Agile (and plenty more that Agile is retroactively claiming credit for), but writing like this turns me off: >Henrich still had a process where success was measured by reports, not outcomes. It was the same reporting process used to measure projects that used linear, step-by-step Waterfall. What are reports, if not documentation of outcomes? What are outcomes without evidence that those outcomes happened? I can see what he's saying, but the more accurate description of the problem "the reports included unnecessary and distracting information." But that doesn't make as good of a blog post as casting it as "paperwork vs understanding outcomes." The whole thing feels like it needs to invent a "paperwork = evil" boogieman just to allow a process change that throws away useless paperwork and allow a team to focus on the paperwork that you just don't call paperwork anymore. Email is paperwork. Version control is paperwork. Change management is paperwork. It doesn't matter if it's software updating a database instead of a secretary updating 3-ring binders. Most of development is paperwork. In fact, online databases mean we do way more paperwork. If we were bottlenecked by that sea of file cabinets in the basement containing all our the legacy projects, we would have way less paperwork just out of necessity. Standups are paperwork that just happen to not use paper.

igouy 7y ago
> Standups are paperwork that just happen to not use paper. If it isn't written down, it didn't happen! (Memory is not an infallible record.)

rezeroed 7y ago

"Sneaks"? There are projects where the agile coaches, scrum masters, and consultants proudly declare regularly that the project embraces the best of both agile and waterfall. Nothing resembling sneakiness.

baxtr 7y ago

AgileFall is coming!

rhombocombus 7y ago

I am currently leading a dev team on a project that I would describe as waterfagile/agilefall. It's a highly regulated industry and the product is for a state level client that has rigorous specifications. We operate iteratively, but there is a discrete golive and many specifications of the project were defined contractually before we pushed a single line of code. For this kind of project it doesn't make sense to operate in a pure agile fashion, but we also have broad license to operate how we please in some aspects of the project (i.e. the UI), and those more closely resemble an agile method project.

jcims 7y ago

We're struggling, IMHO, with adopting agile in a highly regulated environment where there are a multitude of tollgates to get to production. Roughly 20 different stakeholders that need to be accommodated while leveraging many internal services that do not support automation. Slipping back into waterfall can happen because many of these issues have sequential dependencies and are one time request driven processes. It's a bit of a nightmare.

kelnos 7y ago

I've heard -- and much prefer -- the term "Waterfragile". It's funnier, and IMO more descriptive of what's going on.

NateEag 7y ago
Agreed. I was quite disappointed the day it came to me, I searched for it, and discovered I was not the coiner.

eagsalazar2 7y ago

Obviously not every detail is included here but I did find it odd that, although he mentions a focus on outcomes, the change he talks about is entirely about shifting from one process to another, both of which by themselves might be pretty arbitrary. > reviews would be about frequent delivery, incremental development and how leadership could remove obstacle I would think that in a Lean process that outcomes and therefore reports would be focused on performance of North Star Metrics, KPIs, comparing those against goals, analytics/research to help understand that performance, and hypotheses/strategies for how to improve. I'm a big fan of Lean/BML approach and I'm assuming this is what is happening but the article doesn't really send that message. Because of that omission, after reading but before seeing the author my first thought was this was consultant blather.

trentnix 7y ago

It seems every time I read an article where some Agile consultant extols the evils of waterfall the author commits a Just In Case Fallacy or the lede is buried. This article is no different: Teams only checked in – wait for it – every three months in a formal schedule review. I listened as Henrich mentioned that the teams complained about the volume of paperwork he makes them fill out for these quarterly reviews. And he was unhappy with the quality of the reports because he felt most teams wrote the reports the night before the review. That's not a problem with AgileFall or ScrumFall, that's ineffective management irrespective of how the sausage is made. But nobody has to wear the Scarlet Letter of Project Failure if we assign blame to the process! I've also never found that pure (like the wind-driven snow capital-A) Agile matches reality. After spending more than a decade wrestling with it - I still haven't figured out how quality assurance and test fits into the same iteration in Agile. Most teams I inherit have non-technical QA resources. Most projects I inherit have complex business logic and integration needs that aren't easily testable until some state of development "done" has been achieved. And time and time again I find myself staggering their effort slightly behind development so that test and development resources aren't taking turns sitting on their asses waiting for the other side. I've written about this before as was even been publicly admonished by an Agile Manifesto signatory (hallowed be their names) that I was doing it WRONG WRONG WRONG! But I was never able to figure out how to contort reality to satisfy their criticism. Believing in iterations and backlogs and prioritization and shortened feedback loops wasn't enough. Thinking QA - any QA! - occurred AFTER any development was done was altogether impure. And despite asking repeatedly for concrete examples of how its done the "right" way, the only responses I received were vague platitudes and consultant sales blurbs that I guess I wasn't smart enough to understand. In the end, I don't think there's any escaping that work has to be done in a sequence: identify a need, identify a solution, implement the solution, validate the solution. And so every software project has at least a little bit of waterfall mixed in.

Groxx 7y ago
My Grand Unifying Theory of Explaining Agile Proselytizers™ is "if it's not working, you're doing it wrong". True Agile always works. Oh, and it has more-frequent check-ins/iterations than Waterfall (which is the defining aspect of Waterfall: fewer check-ins/iterations than Agile). The actual frequency is undefined, it exists only in comparison to Waterfall. --- The problem is that that kind of stance does nothing but fluff up agile consultant's resumes. It doesn't actually lead to improvements that last, or even discussion that might lead to improvements. Agile is perfect because it's Agile.

bitwize 7y ago

I've never seen an Agile implementation in the wild that wasn't a game of Mornington Crescent, whose stated goal was closer adherence to Agile principles but whose actual goal was to reinforce existing power structures and undermine developer autonomy.

codeulike 7y ago

When feeling troublesome I like to endorse Agile enthusiasts with the skill 'Waterfall' on LinkedIn

api 7y ago

I hate when buzzword sneaks into buzzword.

LifeLiverTransp 7y ago

Power corrupts all methods. Power needs (informal) hierarchy, hierarchy needs communication and coordination limitations. In agile, the customer, usually a arcane diety, shrouded away by requirement engineers and the management, is actually part of the team, and lets face it, the ultimate subversion to all management power. If you can convince the customer representive, that this or that feature is the most important, all the management talk is in the icebox. Which can bring out all the ugly parts in agile. No longterm thinking, just shorterm fast movement, with architectue problems later in the game. So management even has a reason, to persist on "communication" channels. Which is - when agile comes to die in long meetings, made filled with warped perception bubbles. Use Contract First Design and try something wild - a approach that allows your paradigm to pulse between nearly agile and "waterfally" repair-cycles to the damage done to the longterm goals. And be aware that there might be diverging interests- of which none are evil.

bastijn 7y ago

Coming from the other side we have Scaled Agile Framework (SAFe) [0]. Which introduces some waterfall to Agile instead of the other way around. 3 month planning events, commitments, etc. Not bad perse if you work in an environment where not everybody is agile or every process can be agile. [0] https://www.scaledagileframework.com/ https://www.scaledagileframework.com/

effnorwood 7y ago

*AgileFail