9 ms·

I've never quite understood, over 30 years of experience in software development, what problems Waterfall presented that warranted a complete refactor of softwa

• jason-phillips 5y ago
  > I've never quite understood, over 30 years of experience in software development, what problems Waterfall presented that warranted a complete refactor... Because layers of management who need to continually justify their existence can never, ever keep themselves from putting their big fat thumb in the pie and fucking everything up.
• littlecranky67 5y ago
  The problem with waterfall to my experience is not immediately visible to developers or directly concerns developers, but rather the stakeholders and managers. Almost always, your customer doesn't know what they want - they think they do, but they don't and will only realize this after you finished development and have a visible system demo/field test. Additionally it is really hard for project management to get metrics/early warnings if something is not going properly (no project - waterfall or agile - is without problems during its lifetime). How would you asses the quality/progress of a project based on a couple of a specification or requirements documents 1-2years into the project with no (immediately usable) code being available? How would you asses if the approach taken after 3years in a project will be accepted by your endusers? Waterfall is just a big black box to management and stakeholders with the hope that after 5years something usable will come out of it.
  • ivan_gammel 5y ago
    This problem is not a problem of project management methodologies, but rather a problem of business strategy and scope management. If you want to fly to the moon with one shot, that is exactly it. Breaking down scope into smaller and better understood objectives is what will make things work and that should happen before a project starts. Afterwards it can be executed both as one big waterfall sprint for each objective or in some agile way with much higher chances for success and lower costs for cancellation.
    • littlecranky67 5y ago
      Perfect example of unclear formulated goal of the project. You say "fly to the moon with one shot" and think it is completely defined. But the whole, year-long project would change in scope whether you mean "Land on the moon unmanned", "Land a man on the moon", "Land a man on the moon AND return him safely to earth" or "Flyby the moon, get close to surface and take some pictures". And if you now accept that any of those above goals may be changed into another during project runtime, you understood why waterfall is not often the best idea.
      • ivan_gammel 5y ago
        >Perfect example of unclear formulated goal of the project That was exactly my point. >And if you now accept that any of those above goals may be changed into another during project runtime, you understood why waterfall is not often the best idea. I think you missed the entire idea of my comment. There is no sense in accepting that those goals will change and trying to start a project that aims to achieve one of them. They are too ambitious and broad in scope and need to be broken into smaller pieces (see Apollo program or SpaceX roadmap - both are focusing first on smaller objectives, that will clarify the next steps).
  • deleted 5y ago
    [deleted]
  • hamasho 5y ago
    Additionally, as a start-up, we rarely know what the product should be. We have a first vague idea of the product but don't know what it exactly is. Commonly, the ideal product is different from the initial design, and we only know that after the actual design, investigation and development. If it's the nature of software development, the development framework should be capable of design change in short terms. Agile development styles are trying to solve it. Unfortunately, many projects failed to adopt it because they don't understand the purpose of agile soltware develoment.
• Iv 5y ago
  I was once explained the interest of quick iteration cycles (the main opposition, IMO, to a waterfall model) in a simple way: The teacher drew a very simple chart : y = t. "This is, in a given project lasting from t=0 to t=1, the amount of practical information that you have about how to design this project. At t=1.0, you have 100% of the information. When you start, you have about zero information about it, just guesses. Then another line : y = 1-t. "And this, is the amount of design freedom you have during the project. At the beginning, everything is possible but at the end, big changes can not be made." "This is what we call the curse of project management." It has really be enlightening. Make prototypes, be courageous enough to scrape entire designs, and when you have the resources for it : make a total reset for version 2.0 and redo v 1.0 correctly. Of course, this is highly subject dependent. You don't build a bridge the way you build an experimental robot, but this explained well the interest of non-waterfall models.
  • boffinAudio 5y ago
    Waterfall can be fast. In fact, done properly, it is the fastest technique of them all.
    • rubidium 5y ago
      This. If you properly understand the problem. If your team is compact enough to be in the same room. And your customer input during concept/requirement writing is high. If you stray outside that, both waterfall and agile struggle… but waterfall has the potential fatal flaw of delivering a product no one wants (or doesn’t work). This is why agile was introduced. There were software products being made that ended up just not working… so people saw that problem and tried to fix it with agile. But agile just fixes the “don’t deliver a product that doesn’t work” problem.
      • regularfry 5y ago
        It's more subtle than that. What "agile" does (both Scrum and XP, historically) is _protect the delivery team_. With waterfall-style project management, early errors balloon but often don't surface as something that needs addressing until implementation and testing, so the delivery team gets all the stress for blowing out the project schedule. Agile techniques both surface those errors early, so they're course-corrected quickly, and provide a set of clear rules for the rest of the organisation to abide by which should mean that the delivery team can't get overloaded into a deathmarch.
• belter 5y ago
  You can't - Get Certified in Waterfall. - Claim to be a Waterfall Master. - Waterfall Standup sounds like a song.
  • pjc50 5y ago
    You absolutely can get certified in various waterfall processes: https://www.amazon.co.uk/Rational-Unified-Process-Reference-Certification-ebook/dp/B0050H3VNG https://www.amazon.co.uk/Rational-Unified-Process-Reference-... (I can't believe they chose the "when all you have is a hammer everything looks like a nail" cover!)
    • belter 5y ago
      RUF was never about Waterfall. Instead was the first effort to progress from there "(Rational) Unified Process vs Waterfall Model" https://stackoverflow.com/questions/20560514/rational-unified-process-vs-waterfall-model https://stackoverflow.com/questions/20560514/rational-unifie...
  • froh 5y ago
    Automotive SPICE certifications would like a word...
  • commandlinefan 5y ago
    > Waterfall Standup sounds like a song TLC actually did record a (correct) song about it: https://www.youtube.com/watch?v=8WEtxJ4-sh4 https://www.youtube.com/watch?v=8WEtxJ4-sh4
  • ziggus 5y ago
    Well, considering that Scrum Master certification can be done in a day, I'm not sure I'd be crowing about it.
• frant-hartm 5y ago
  > There is a natural process that every issue goes through: Analysis, Specifications, Requirement, Design .. then the Developer takes in the materials of each of these steps, does an Implementation and Testing phase .. then the issue has to be validated/verified with a Qualification step, and then it gets released to the end user/product owner, who sign off on it. If you do this feature by feature than you have described iterative/incremental development process. Scrum is one example of such process. Yes, Scrum is many little waterfalls. But with Waterfall you Do the analysis of the whole project/product, check Specification of the whole.., check Design, check Implementation, check Internal testing, check UAT - this is the first time the customer sees the product, it's too late. If you think it's stupid to do it this way you are right, that's the point. If you haven't seen this in real world then you probably didn't work on a government project or you were very lucky.
  • boffinAudio 5y ago
    I've worked on all kinds of projects, and every single time the project failed, the tooling and methodology were blamed, not the individuals. This is the #1 cause of failed software projects - the inability for individuals to take responsibility for parts of the workflow, for which they are incompetent, and not subsequently working on improving that competence at an individual level. But true, breaking larger problems down into smaller units (Agile/Scrum) is an effective way for a "manager" to help their developers - but I argue this is non-optimal for sociological reasons, not technical: Adherence to a cargo-cult is a function of laziness, not competence.
    • lupire 5y ago
      > Adherence to a cargo-cult is a function of laziness, not competence. Why doesn't this claim apply to Waterfall?
    • AnimalMuppet 5y ago
      So when waterfall doesn't work, it's the fault of the developers, who are doing it badly? Well, I've seen agile work, too, with really good developers. And plenty of people have seen both fail, if you don't have good enough developers. Hmm, maybe it's not the methodology?
• nicoburns 5y ago
  "Agile" processes still have all these steps, they just repeat them in a loop on a much smaller timescale. This allows for a lot more flexibility, and for feedback from the qualification of the first iteration to go in as input to the analysis stage of the next iteration.
  • hamasho 5y ago
    I agree with that. But it's worth noting that we need to spend more time on task management with a shorter development cycle. In waterfall or scrum or whatever, one development cycle often comes with listing tasks, prioritizing, planning, developing, retrospection, or something similar. If the cycle is long, we can spend more time on each process. But if the cycle is short, we rush to complete each process and can't spend enough time understanding each process's meaning. So I focus more on consuming as many stories as possible. I never think about the product's goal and how I can contribute to it. So we have to spend more time teaching the purpose of each process. Or we end up as a failed agile development.
    • jayd16 5y ago
      Sure, but that's kind of tangential isn't it? You can just churn through tickets thoughtlessly in both methods, no?
• CraigJPerry 5y ago
  Royce argued that only in small software systems can you do a little bit of analysis then a little bit of coding then be done with the project. He proposed waterfall as a way to build larger software systems. He argued that these 2 steps were the only direct value steps in developing software - figure out what the problem is then write a little code to solve it - he acknowledged this was the perfect ideal since the two steps directly benefit the output product but he also argued this can’t scale to larger systems. So various analysis and testing steps were added (none of which directly contributed to output, all were drags or costs on delivery - with the idea that they catch more problems earlier thus paying for their overhead), with feedback between them to catch earlier errors, and thus waterfall. The agile mindset revolves around the idea that the other steps added are BS. The agile mindset solves for the impossibility of a 2 step analysis then code process at scale by reducing scale. It does this by using very small iterations and removing handovers between different parties.
  • boffinAudio 5y ago
    Agile has failed the industry precisely because the other steps are NOT BS. Unfortunately, de-programming Agile-adherents is more work than actually just doing those natural, proper steps.
    • drran 5y ago
      Agile is just 20-30 small waterfalls instead of big one. How small a waterfall needs to be to fail?
      • boffinAudio 5y ago
        Honest opinion: Its not a matter of size. Its a matter of completeness. Agile or waterfall processes fail if you omit the Qualification step, or if you don't ensure that the analysis produces actionable requirements, or clearly formulated specifications that lead to requirements, user/technical/or otherwise.
        • 5y ago
• ImportOllie 5y ago
  I always saw agile as essentially breaking waterfall into mini waterfalls.
  • hvgk 5y ago
    As I call it in our process extreme waterfall. We do the bad bits of both sides of the fence and forget the good bits.
• Scarblac 5y ago
  > There is a natural process that every issue goes through: Analysis, Specifications, Requirement, Design .. then the Developer takes in the materials of each of these steps, does an Implementation and Testing phase .. then the issue has to be validated/verified with a Qualification step, and then it gets released to the end user/product owner, who sign off on it. I've never worked at a place where all these things actually happened. Everything was responsibility of the developer, who would complain about the absurdity of that. Then management eventually introduced Scrum, as a way to excuse their behaviour. "We don't need an analyst, we use Scrum now."
  • boffinAudio 5y ago
    Developers who fail to take responsibility for the full workflow, fail. Its not "Agiles" fault, although this is often used to justify the failure. Developers have got to realize that they are responsible for the full workflow from beginning to end, and only poor/low-quality developers will work to change that natural law - with negative effect.
    • Scarblac 5y ago
      Imo analysis needs domain knowledge, the developer can do it on their own but it won't be as good as when a domain expert does. A developer doing their own QA will have the same blind spots as during development. And of course, if he was also doing the analysis, he'll have the same blind spots as during analysis.
      • boffinAudio 5y ago
        Yes, waterfall is a team activity, as all software is necessarily a social service. Developers that fail to understand this - or indeed, resist it as part of their cultural identity - usually get taught this lesson hard in the form of failed projects.
    • pjc50 5y ago
      > Developers have got to realize that they are responsible for the full workflow from beginning to end That is not what the org chart says.
• flohofwoe 5y ago
  > .. then the Developer takes in the materials of each of these steps IME this is the main problem, the "Developer" must be heavily involved from step 1 (and work in a close feedback loop with QA). Everything else follows automatically in ever smaller iteration steps. Software development is first and foremost experimental research work, not a factory line. E.g. if a specific software development task feels like boring/repeating factory work, it should have been automated already.
  • boffinAudio 5y ago
    Yes, developers and managers have to be involved in the full workflow. Why is this so hard?
    • lbriner 5y ago
      It is hard because more people means more opinions. * Involving the developers in everything that the product team does as well as all of their own work is too inefficient. * Involving the wrong developer means you might not ask the right questions up-front. * Some developers are too negative and block things early on * Some developers are too positive and will say yes to everything even if it is unreasonable * There is not always a clear authority between product owners and developers * A lot of decisions are based on company priorities that might need someone outside of the product/development team to argue for
      • boffinAudio 5y ago
        > more people means more opinions. I'm sure 'manager people' have their own processes for this problem.
• jollybean 5y ago
  ??? It's predictability. You can plan the construction of a building, because the elements are known. If someone fails to deliver a bag of nails on time, it won't matter. If you add labour to a project, it will very ballpark accelerate linearly. 1 team, 10 houses = 10 weeks, 2 teams 10 houses, = 5 weeks. Ballpark. More developers do not mean the software will be finished more quickly. There are economies of focus in software, generally not scale. Which is why you want good developers. The other factor is that 'Requirements Change' in Software more than they do in building, and that's the other #1 issue that causes problems. The entire ethos of 'small releases, often' is built around that. Software is evolved a big organically for this reason. Waterfall is obviously counterproductive. If there is a 'default' methodology in software it's 'Iterative Waterfall' whereby you break the project down into the smallest reasonable phases. There can be an overarching plan but it has to be nimble.
  • drran 5y ago
    > 1 team, 10 houses = 10 weeks, 2 teams 10 houses, = 5 weeks. Ballpark. Concrete needs 4 weeks to cure.
    • iso1631 5y ago
      https://www.hunker.com/12002648/how-to-paint-an-exterior-house-foundation-with-color-schemes https://www.hunker.com/12002648/how-to-paint-an-exterior-hou... suggests building doesn't need foundations to be 100% cured, and you can start after a week or two Here's exactly the same article but translated for a UK perspective, it's hilarious https://www.ehow.co.uk/about_4674117_were-concrete-blocks-first-made.html https://www.ehow.co.uk/about_4674117_were-concrete-blocks-fi... > Do not pour concrete in temperatures below -6.67 degrees C. > Temperatures of 21.1 degrees C compared to 10 degrees C, for example, reduce the 50 per cent strength curing time from 6 to 4 days
      • jollybean 5y ago
        The issue is not cement or how long it takes to be cured, rather the crude and easy determinism of it all. Home builders don't generally face existential struggle with the timing of the issue. You don't go 5x over budget and have it take 5x longer because of this. You can reliably schedule around cement.
• g051051 5y ago
  > I've never quite understood, over 30 years of experience in software development, what problems Waterfall presented that warranted a complete refactor of software development processes such that things are now very much in cult-/cargo-cult territory. It's refreshing to see someone else make this point.
• pydry 5y ago
  The problem is that the more unknowable the future is, the quicker your plans about the future will be invalidated. Thus a quick iterative feedback loop where there is a tight lead time between user feedback and users using the software tends to work better because it lets you adapt much more quickly to changing circumstances. This is why I always aim to minimize that feedback loop as much as is feasible. Unfortunately you are right that it got bound up in a pseudo cult. Worse, the cultish practitioners tend to do as all cults do and put their emphasis on ritual (e.g. standup) and completely miss everything else. I kind of hate the term agile now precisely because the movement almost set out to create a cult out of this.
• bregma 5y ago
  Coders just want to code. Waterfall means you have to wait before you code, and plan, and organize. Agile means you just start coding, and code and code and code until you leave and someone else has to clean up the mess. This is no different than most other activities in life. See the Stanford marshmallow experiment.
  • lbriner 5y ago
    So you are saying that agile happened because of coders and not because there were fundamental problems with using waterfall on large projects? No part of agile says that you get rid of planning and organizing, it is just done in smaller slices in shorter cycles. Having used both over 25 years, I wouldn't look back at waterfall for any project except the very smallest. No-one allows you to lock some requirements in for 5 years any more, something that was accepted back in the day. I have plenty of examples of waterfall projects that delivered a number of things that simply weren't required any more and that was a failure of a long-tail project plan. Agile also allows you to work iteratively on a project that is never finished i.e. SaaS, which is not possible with waterfall.
  • drunkpotato 5y ago
    I think this is half of it. From my experience, the other half is that "agile" is a wonderful excuse for managers and product managers not to have to plan anything or commit to any priorities, and for software companies not to hire project managers. I don't think this is particularly agile's fault; no process can fix broken organizations, and most tech organizations are broken. Luckily the magic money machine papers over grotesque inefficiencies so we can all be gainfully employed without having to become real professionals!
• pjc50 5y ago
  > what problems Waterfall presented These are presented in the article. > Analysis, Specifications, Requirement These tend to be inadequate, incomplete, or .. > Qualification .. at this point the customer discovers that what they asked for does not actually solve their problem. You cannot do any kind of exploratory or innovative work in waterfall, because you can't specify that upfront. You can't easily co-evolve solutions by discovering what works and what doesn't. You can't even do A/B testing really, because every time you do your GANTT chart bifurcates. There is a history of big, spectacular IT project failures, often commissioned by the public sector, but not always. The evolution of non-waterfall systems has been driven by trying to prevent or mitigate these failures.
  • boffinAudio 5y ago
    >> Analysis, Specifications, Requirement >These tend to be inadequate, incomplete, or .. >> Qualification >.. at this point the customer discovers that what they asked for does not actually solve their problem. I disagree. If you are properly isolating your work into an "Analysis" phase, you will properly flesh out the issues and the subsequent specifications and requirements will be complete - sufficient to the task of formulating a development/implementation plan. But if you don't take care to complete a proper analysis - then no, you won't formulate specs or req's properly, either. The important thing is that the individuals involved know the value of a proper analysis - whether they are developers or managers or just paying the bills. Too often, software developers don't realize that they are really providing a social service, and thus they don't deliver proper analyses that can result in action. Alas, that is something other than a tooling/methodology issue and lays more in the motivations of the developer. Its a lot easier to say you can't be responsible for the mess as a developer if you don't allow the mess to be adequately described, in the first place .. so the Agile vs. Waterfall debate is really more of an ethics issue than anything else. Good developers do Waterfall Complete. Mediocre developers do it, and call it Agile. Terrible developers find excuses for not doing the key things, and end up with a failed project on their hands.
    • pjc50 5y ago
      > If you are properly isolating your work into an "Analysis" phase, you will properly flesh out the issues and the subsequent specifications and requirements will be complete - sufficient to the task of formulating a development/implementation plan. But if you don't take care to complete a proper analysis - then no, you won't formulate specs or req's properly, either. But what if that doesn't happen? (This seems to parallel the "C is safe if you don't write bugs" argument. Processes that require infallibility are bad processes.) The level of dysfunctionality that can be achieved is baffling. How can someone fail to deliver a payroll system? https://www.smh.com.au/technology/worst-failure-of-public-administration-in-this-nation-payroll-system-20130807-hv1cw.html https://www.smh.com.au/technology/worst-failure-of-public-ad... - it's one of the oldest applications of business computing, and yet somehow it wastes a billion Australian dollars?
• theshrike79 5y ago
  If it helps you can think of it like this: Scrum is just Waterfall, but with faster cycles. Traditional Waterfall(tm) is when you spend a year in analysis, a year in specifications, a year in requirement, a year in design... And then you put out a piece of software that's already outdated and doesn't do what the customer wanted or needed. But the company got paid anyway, so off to the next project. With Scrum you go through the whole waterfall loop every 2-4 weeks and the customer and the team have a chance to amend any of the phases after every iteration. This way there's less of a chance the product is completely outdated and unnecessary after the whole project (comprising of multiple waterfall loops) is done.
  • zoomablemind 5y ago
    >...And then you put out a piece of software that's already outdated and doesn't do what the customer wanted or needed. Or software that was built on outdated foundations, like frameworks and providers which may have been "in-vogue" at the product conception, but deflated or advanced into maintenance-only stage by the product release. Let's not forget that the competition is not sleeping meanwhile, so Waterfall is equally capable of pushing out half-cooked products just to be ahead or to meet obligations/expectations.
    • theshrike79 5y ago
      Exactly. I've seen this in military contracts where the rules are REALLY strict (and ancient). Everyone on the project knows at one point that the resulting product will be outdated and useless, but neither side can call it quits. The one making the product needs to finish or they won't get paid, the one ordering said product can't cancel the project or they'll get sanctions according to the contract. So they just go through the motions and produce something that'll never get used and is put straight into the bin.
  • randomdata 5y ago
    On paper. In reality, Scrum is waterfall with a daily standup for management to keep track of progress.
• Kim_Bruning 5y ago
  Here's one way to explain it: The generalization is that waterfall is a form of organization level open loop control and scrum is a form of organization-level closed loop control of your process. In one case you set a goal and then move towards it (and "damn the torpedos"). In the other case you stop and look where you're going and adjust course at regular intervals iteratively (But "are we there yet?"). In real-time applications, open loop control only works well in narrowly defined situations, but nevertheless can be very useful to get somewhere in a definite amount of time. Closed loop control tends to be more robust in that it is rather more certain to get you to the target in a wider range of situations and in the presence of noise. The downside is that it can be harder to determine when you'll get there. This way of looking at it, you immediately see the balancing act going on. Attaching specifically to what you are saying: if you do the Analysis, Specifications, Requirement, Design, Implementation, Testing, Qualification, Signoff just once, your product-objective fit will only be so good. (This is where waterfall stops and calls it a day. It's just a very long "day") Once a change is signed off; review it in production for a while, take lessons learned, and reapply them to a new Analysis step (and move forward through the steps again). After the second sign off, in real world practice you're likely to have a much better product market fit than the first iteration. Wash rinse repeat, after the third sign off, you'll find an even better fit. This much should stand to reason. You combine your original plan with everything you've learned. Then take the new plan and add in everything you've learned, etc. How could your 5th, 6th or 7th, Nth version NOT be better than the first one? This is closed loop control. If I present it this way, it would seem like open loop is always going to be faster than closed loop. Well, in biology this is actually exploited as such: fast processes are often open loop. Fortunately it turns out that if you're doing closed-loop anyway, the self correcting properties of uberhaupt having a closed loop means that you can get away with cruder/faster steps in the loop. As long as you keep the loop closed. This stands to reason, right? (And else it stands up to empirical testing.) If you're in a hurry, it might be the case that it's ok to merely hit home close to the target, rather than bang on 100%. Despite some theoretical slowness, you might be able to get away with using a closed loop process with fewer/cruder iterations to get to your target more quickly in practice. Now what a lot of cargo-cultists do in the case of open loop (and waterfall) , is that they apply it to very long projects, but forget that the world might change while they're still working. Open loop works best for short, well defined projects. In the case of closed loop cargo cultists: They might end up taking cruder faster steps, but then don't actually close the loop by going back to the start and iterating. Now you have a very bad product. In practice you'll need to tune your process to your own needs.
• zoomablemind 5y ago
  > ...There is a natural process that every issue goes through: Analysis, Specifications, Requirement, Design .. then the Developer takes in the materials of each of these steps... What do Developers do before it's a green light for them to start the dig? What do Designers do before it's "their step"? What do they do afterwards? What about those in preceding stages? Well, they might as well do whatever else next that falls on their seats, but sure everyone still cares to see the "final output", that is the developed and tested product. Yet it's inevitable for them to switch their context to something else meanwhile. And this detachment is what makes Waterfall process often protracted and resistant to change. Not speaking that there's also a Customer somewhere in that pipeline, hoping to derive some value and feed some form of input. Some industries/companies/teams/products may be just ok with waterfall or no methodology at all. Other situations may want shorter cycle or even parallel efforts to stay in line with demand/constraints/risks. Also, there's a "working software" factor too. Call it a PoC/proto/early demo... It not only tests the ideas, it also motivates to keep going or offers assurances that it will be done "right". If anything natural there is to any process is that most professionals want to do their jobs well and be proud of their effort, preferrably rewarded for the success and not singled out for blame of failures. The question is then how to create such an environment to make that possible. Agile on not, what cargo-cults it is the general lack on internal vitality in some industries/companies/teams/products. And as such it affects the very professional values on all levels, eroding the collective "care" about the result or even the process itself.
• unclebucknasty 5y ago
  I hate to admit it but, In my experience, agile has brought productivity gains. When practiced correctly what agile seeks to do is, not skip the steps you mentioned from the traditional waterfall model, but to apply them iteratively to individual feature sets in sprints, versus across the entire project. I have seen many products fail to meet requirements and dramatically slip schedules under the waterfall approach. The project becomes too large and unwieldy, and too many assumptions are made that create dependencies for future features. The reason I hate to admit agile's success is because it comes at a cost--namely there are frequently externalities such as lack of documentation and developer burnout. Many agile implementations treat developers as cogs on a production line, responsible for essentially participating in automated CI/CD processes as a piece of software might be expected to do. And the relentless unending sprints can also easily take a toll.
• OneTimePetes 5y ago
  Its the symbolism- returning to a earlier stage of planing. Take building a house- the moment you have to redraw the plans, the previous work is usually a tear-down. Often enough - that is the case in the real world. Berlin Airport, was a waterfall project, were several times additions were added, late in production. Now that image carries only halfway into the software world, as alot of the other abstractions (classes, modules, interfaces) will survive, even a replanning and rewrite in waterfall.
• tyingq 5y ago
  I've been doing this a long time. As far as I can tell, it comes down to senior leaders in software development wanting to make some sort of success story to further their own careers. It's much easier to buy into somebody else's canned success story plan. Hence the popularity of stuff like "Scaled Agile". Which is funny to me, because in practice, it's heavier than old school MS Project and Waterfall development. Even the diagram they use to summarize the concept is laughably undecipherable and complex[1]. Same reason companies latch onto 6-sigma, Clifton strengths, TOGAF, Total-Quality-Management, and so on. I do recognize there is some actual "good stuff" in there. I'm a fan of the textual agile manifesto. [1] https://19yko92jjsfl3euk0g1esoja-wpengine.netdna-ssl.com/wp-content/uploads/2021/01/SAFe_5.1_Full_SAFe_WEB.png https://19yko92jjsfl3euk0g1esoja-wpengine.netdna-ssl.com/wp-...
  • deleted 5y ago
    [deleted]
• dec0dedab0de 5y ago
  There is a natural process that every issue goes through: Analysis, Specifications, Requirement, Design .. then the Developer takes in the materials of each of these steps, does an Implementation and Testing phase .. then the issue has to be validated/verified with a Qualification step, and then it gets released to the end user/product owner, who sign off on it. Doing that per issue is still basically agile. The problem is when you do it per project, and try to get everything right the first time, and then it's months or years between analysis and qualification. Agile really boils down to starting small and keeping your stake holders involved. It is an explicit acknowledgement that programmers are bad at estimating time, and that users don't know how to ask for what they want. All the stuff people complain about is just companies buying snakeoil, and overzealous organizers justifying their own jobs.
  • felipellrocha 5y ago
    That might be its stated goal, i still haven’t seen a single shop that actually implements that. First thing that happens when implementing agile are estimations, and the justification for that is for “predictability”.
    • fknorangesite 5y ago
      > First thing that happens when implementing agile are estimations I'm usually the first to chime in that all estimates are lies, but small estimates (i.e., per-issue, not per-project) easier than big ones.
      • ineptech 5y ago
        It really helps to use vague terms like S/M/L/XL or bike/car/plane or whatever. If you say an epic will take 6 months, it is very likely that in 3 months someone will ask for proof that you're halfway done.
        • cpeterso 5y ago
          But once you have abstract S/M/L/XL estimates for some tasks, what do you do with that information? How many S tasks are equivalent to 1 M task? If the team completed 3 M tasks and 3 S tasks last sprint, how does that help you plan for your next sprint? While story points have their own issues, at least tasks' relative size differences are clear and different tasks of different sizes can be scheduled.
• brightball 5y ago
  Planning beyond 90 days out and pretending the plan has any hope of accuracy, that new information won’t change the plans (or design) and that the plan you started with will still even be beneficial when you finish. Brittleness. Dependency planning has to happen of course. That’s not unique to waterfall, it’s just planning.
  • bgroat 5y ago
    I agree with this. But why not waterfall out 90 days then? Or 75 or something
    • brightball 5y ago
      You’re basically describing Scaled Agile. They have big planning meetings every 8-12 weeks (called PI Planning) where devs make a plan based on priorities, work out dependencies and get agreement with the business side of the house about the plan itself. In my opinion, it’s the ideal balance of concerns.
    • Jtsummers 5y ago
      "Waterfall out 90 days" is not what is derided as Waterfall. The derided form of Waterfall is for large applications and commitments, 90 days is small in context. That's the prototype phase of a project that Waterfall may be applied to. The basic model of Waterfall is, indeed, fine if you iterate, which then isn't Waterfall. The basic model + iteration + shorter timeframes (probably under 90 days, certainly under 180) is basically just the same plan-do-check-act cycle from Deming (and others). You want it to be shorter so you can respond to new information (including the results of the partially developed system). Waterfall, the derided form, doesn't give you feedback until final delivery, or maybe the phase called "FQT" (final qualification testing). Importantly, until the customer is re-involved (which may happen in FQT or in delivery) you don't get revalidation of the system. System engineers learned and applied this as far back as the 1950s, at least. No serious large scale system uses Waterfall and reliably succeeds.
• gilbetron 5y ago
  Traditional waterfall model doesn't allow for iteration between the main steps, and this was followed historically, and caused absolutely massive delays and cost overruns in projects through the 70s, 80, and 90s. Popping iteration onto Waterfall model and still calling it Waterfall model is being disingenuous.
  • Jtsummers 5y ago
    And cost overruns and delays in the 00s and 10s! People here on HN keep telling me I'm imagining things, that Waterfall either isn't bad or isn't real. But a 9-figure project I was (fortunately briefly, though not brief enough, around 1 year) on was several years late and delivered only half the features the users needed. Why? Because they made the plan in the mid-00s, and the circumstances changed but the plan didn't. They did what Waterfall describes, they committed to their initial analysis and plan and never looked back. Fortunately they learned after that, but it was a costly exercise.
• karaterobot 5y ago
  For what it's worth, I've never been on an agile team that did not have some waterfall decision making that was not open to correction, or a waterfall-based process that was totally closed off to testing and making changing along the way. I think we argue about platonic archetypes that don't exist in reality. Most processes and teams I've worked with have resembled each other more than not, if you ignore the terminology they use. I think success or failure of a project comes down to a lot of factors that are hard to quantify, so in some sense our focus on process methodology is a bit ritualistic and cargo culty.
• Jtsummers 5y ago
  Yes, that is the natural process. However, Waterfall (big-W) is a Big-Bang Big Design Up Front approach, each phase is months long and after the analysis you do little or no revalidation (emphasis on re) and a release planned 1-5 years out. That's where the problem comes in. If you revalidate the plan and change the plan, you're no longer doing Waterfall. You're doing something else, you're using your brain. If you break that natural process down and apply it to subcomponents and have releases every few months, you're doing something like either the classical Iterative & Incremental or Evolutionary models which are longer cycle versions of what Scrum aims for (which as a target tries to get you under one month, maybe down to 1-2 week cycles). I&I and Evolutionary tend to operate with cycles from 3-6 months.
• jayd16 5y ago
  Maybe it is natural, but instinct isn't always best. All you need to see is that its obvious that if you don't have continuous design/stakeholder feedback you can spend a lot of resources building the wrong thing. A lot of work is put into preventing this. The cargo culting and snake oil sales exist but its just a fact of life not exclusive to agile methods.
• spookthesunset 5y ago
  "Analysis, Specifications, Requirement, Design". When people hear "waterfall" they imagine doing this for the entire project end to end. It's super risky to do this because nobody knows what they want until some of it is built. And if you just spent a year of a teams time building something and "tada" you show the user... odds are not in your favor that it will do what everybody actually wanted. Plus odds are good it might not work because there was no iteration to shake out the rough spots. To me, the key for "agile" is rapid iteration. It's a bunch of mini waterfalls where every step delivers something of value (even if that value isn't directly visible to the end user yet). Each iteration forms a feedback cycle that helps make sure a team is delivering something the end user actually wants. In practice I don't think anybody is actually doing "waterfall classic". It's more of a story we tell to remind us how important it is to get in the habit of rapid iteration. (Sidenote: there is plenty of other reasons to rapidly iterate. The end project is built on a moving target. Business processes change over time, the competition changes over time, etc... if you have your specs 100% locked in at the beginning you'll find that half of them no longer apply after a year because so much has changed in your environment. There is also inventory costs of keeping so much code out of production for so long.)
• hpoe 5y ago
  So I'll start by saying I detest the snake-oil salesmen professional scam artists that are "Agile consultants", as they have now begun to leach onto DevOps which honestly is in large part has the same end goal of agile. These people are largely talentless hacks and parasites that extract value from coporations that are making money and then move onto the next victim while the actual engineers deal with the infection their ideas have left. That being said the issue I've seen and read with waterfall is that it is very slow and cannot adapt very well. Which honestly is a super great thing in certain fields, like avionics, NASA moon landers, those kinds of things, because with this lack of speed and rigour you also get a certain level of assurance and quality and predictability, assuming things retain a certain level of consistency. However in the world of newly emerging markets and the .com bubble and the new SAAS bubble being slow is a death sentence because at this point it is better to spray and pray because their are plenty of VCs willing to handle you fistfuls of cash to shoot at the wall because they are hoping to get in on the next big Facebook or Google (in reality now they are just hoping you will get bought by FB or Google which is another rant for another time.) That being said agile offers flexibility and adaptability in return for offering less predictability. This causes a huge rift however when you have in house tech resources, that are part of a larger organization that needs all the assurances and predictability promised by waterfall, but the in house resources have to compete against SaaS opponents that management is all to happy to use a credit card to purchase. The end result is tech teams saying they are agile by which they mean they respond to whoever is yelling the louadest and success is determined not by value but instead by politics and in the meantime no one is taking care of the customers because and you end up with everyone's data getting resold a dozen times, having more security holes than an Alabama "don't shoot at signs" sign, and companies spending several million dollars a year to keep some rusting old COBOL application running because it is the only thing they know for sure works right now. Sorry I might've been unloading some baggage in this post.
• SmellTheGlove 5y ago
  > I've never quite understood, over 30 years of experience in software development, what problems Waterfall presented that warranted a complete refactor of software development processes such that things are now very much in cult-/cargo-cult territory. I'm going to tell you up front that I'm a CSPO (certified scrum product owner), and that I don't exactly know. I don't mean this sarcastically - it's more that I'm not sure what agile/scrum fix, versus where would _any_ process change naturally cause an org to address its weak points. Example - For a while I was pretty into scrum - the company I was at transitioned from waterfall and it did give the appearance of faster delivery. In hindsight, I think what it really did was provide explicit authority to a decision-maker (me, or the PO in general), and build a culture of keeping docs updated regularly, and therefore status visible/known. Waterfall didn't break these things, but in the past we were slow to unblock things and nobody really knew where the work was unless they went and asked (and had someone do an adhoc update). I'm now at an org where a team that isn't mine is trying to be pretty strict about scrum, doing all of the requisite incantations and such. The issue is, they have more management than they do actual developers on the team. Adding this process on top makes the management happy, but it hasn't done a thing to boost anything related to development. It's exactly the cargo cult behavior you describe, when IMHO agile is best thought of as a toolkit that you borrow from selectively to fix the things that need fixing. I think going all-in isolates engineers from the people and processes they're trying to help and reduces them to basically short order cooks pulling tickets off the rack. I get that it's meant to make them more efficient, but I think that isolating your most expensive people whose job description says "solve problems" instead of having them engage directly is the wrong move. Mind you, I don't think I'm necessarily right about everything but I've seen enough broken shit to know I'm not totally wrong either. Now as a fairly senior leader, I discourage my teams from going all-in on agile and push them toward looking at their process, identifying what's broken, and fixing that (with agile principles or not). It's rare that layering on more process has a positive effect and I like folks to be thoughtful about those changes.
• dekhn 5y ago
  in my 30 years of software development, everything has been a combination of waterfall and agile and projects that attempt to adhere to one or other strictly end up failing.
• 62951413 5y ago
  I joined the industry only 20 years ago so missed the heyday of waterfall. I did witness the rise of the RUP though. And I must say that to jr developer me it was very educational despite its rather intimidating volume. There's one angle I don't see discussed in the last 10 years or so. There was a time when a lot of attention and literature was dedicated to software design. The whole OOP movement, design patterns, the RUP, entire shelves of software architecture books. I'm inclined to call them systematic approaches to building software. I believe feature-driven development was the last such methodology aimed at developers. Ever since Agile became popular around 2008 I have not seen anything comparable. The OOP school of thought lost its appeal without really being replaced with another well defined body of knowledge. Anything FP-related is extremely community-specific and nobody is really talking casually about, say, monad transformers. Any Agile methodology has very little guidance for developers in comparison with RUP/FDD. It's all about managerial concerns and time tracking. But there's no question that software development is drastically different now. Using version control (even local one) was not a given in 1999. Unit testing with jUnit in 2001 was a major revelation. And shipping features constantly in SaaS got popular after 2007 or so.
  • corpMaverick 5y ago
    I was never a fan of RUP. But it had the idea of adapting the process to the project. It gave you lots of choices. With Scrum, every project is approached the same way, and no one thinks if it makes sense or not. McConnell´s "Rapid Development" also gives you alternatives for you to choose. Now days, it is Scrum or sometimes Kanban.
• tsimionescu 5y ago
  I have about one third of the experience you have, but still I ended up in a place that was doing something very close to the mythical waterfall, and it was a horrible system. We started with a one year long release plan, with a defined set of features that were supposed to make it in. A few weeks were spent defining functional specs for the features, negotiating them with PMs, and finally handing them over to the QA department (which was entirely separate). Then, dev work would start on all these features, while QA would come up with a series of Acceptance tests, Feature Tests, System Tests, Final Tests, Automation test plans etc. - all based on the FS. Timelines and Gantt charts would be produced, deciding how to fit the features and required testing into the 1 year timeline. After many months of dev work, the first builds would start making their way to the QA department, who would start tentatively running the Acceptance tests for the features that were supposed to be delivered. Some rapid iteration on any bugs affecting the Acceptance tests would happen here. Once the Acceptance test pass rate was high enough for the feature, QA would start working on Feature Testing, logging Bugs, while the dev teams would move on to other features. Occasionally QA or other devs would complain about blocking bugs that would have to be addressed ASAP, but otherwise bugs would pile up. This would continue for all features, with dev teams keeping an eye on the total bug counts trying to make sure they would stay below some limits that had been set in a Bug plan (typically this meant keeping below 2-300 bugs for a team of ~5-6 people). Finally, all features would be delivered to QA and passed acceptance, reaching the Feature Complete milestone. At this time, the dev team would start work in earnest on bugs, while the QA teams would continue with Feature Testing, logging new bugs. This would continue for 1-3 months typically, until the Feature Testing would reach a decent pass rate, reaching the Feature Test Complete milestone. QA would now start on Performance testing and Early Final Testing. Often this would be the first time multiple features would truly start being used together. Dev teams would still be on full bug fixing mode. When bug counts were low enough and Early Final Testing had a good pass rate, the product would reach the Code Chill stage - only important and safe bugs would be allowed to be fixed anymore. At this time, Final Testing would start in earnest, with the most complex cross-functional tests. The dev team would be finalizing the remaining bugs, until Code Freeze was reached - the vast majority of testing done, and only critical issues being allowed to be considered for fixing. Of course, during this while, deviations from the schedule devised initially would be monitored constantly, with pressure coming not only from upper management, but also between QA and dev, as delays on the dev side would easily translate to delays on the QA side. Customer feedback on previously released versions of the product, or new opportunities, would be very hard to squeeze in, causing panic and chaos in all the well laid plans. More often than not, it would be left to next year's release, meaning the average time between an opportunity and a release containing a feature to address that opportunity would be ~1.5 years. Dev and QA would often be at odds, blaming each other for schedule slips, complaining about silly bugs, trying to pass the buck. Of course, overall the whole thing worked, but slowly moving to a more Agile working model was a massive relief to everyone and created much better team cohesion, much less pressure on deadlines and deferred features (in the past, a feature getting kicked out of the current release meant it would only come one entire year later at best), and a much better product. Huge issues in design would no longer be treated as bugs, discovered weeks after the initial delivery of a feature. Tests would be designed by dev and QA together, often being run iteratively days after the code is written, leading to much better quality and corner cases getting caught early, not late. Process adjustments are now local team decisions, whereas before they required VP level involvement and agreement and coordination between all teams.
• corpMaverick 5y ago
  I am about as old as you. Unless you started at 13. The problem was that projects spent months doing analysis and design and never ship everything. Also agile recognizes that you don have all the answers at the beginning; you need to experiment and see how things should work. There was also a tendency to treat people as factory workers that you could assign any work and they just convert specs into code. We still do that with Scrum. :(
• earleybird 5y ago
  The best reference I have so far is from 1970[0]. Waterfall is page 2. Reading past pg 2 you will see attributes of agile and various techniques described. The one thing that comes to mind is the blind men and the elephant[1]. Quoting from the wikipedia link (which quotes another): O how they cling and wrangle, some who claim For preacher and monk the honored name! For, quarreling, each to his view they cling. Such folk see only one side of a thing. [0] http://www-scf.usc.edu/~csci201/lectures/Lecture11/royce1970.pdf http://www-scf.usc.edu/~csci201/lectures/Lecture11/royce1970... [1] https://en.wikipedia.org/wiki/Blind_men_and_an_elephant https://en.wikipedia.org/wiki/Blind_men_and_an_elephant