12 ms·

We always end up with waterfall

152 points by KingOfCoders 3y ago

deleted 3y ago
[deleted]
pjmlp 3y ago
Easy, as somehow discussed on the article. Beyond the IT department everyone else has very to low incentives to adopt "those crazy ideas from computer guys", they care about deadlines, roadmaps, when marketing campaigns can be organised, with what content, what should be on the contracts with business partners,... Ever since we got those ideas around 2000, the best outcome if agile is adopted at all, is scrumfall as the article describes. However I have also seen many companies go back to classical requirements documents, and all related processes, as if agile had never been in the building.
- mrmlz 3y ago
  Yeah, because waterfall with iterations is better than SCRUM/AGILE/WHATEVER. God intended water to run from the TOP to the BOTTOM. Don't fight gravity.
  - psunavy03 3y ago
    [flagged]
  - atoav 3y ago
    The best results I have ever seen have been projects that people on the bottom did themselves, because they needed it without the people on the top ever knowing about it. The worst results I have ever seen have been projects where the people on the bottom stopped caring because the people on the top tried to make them ignore the laws of nature. Also: God is dead
    - mrmlz 3y ago
      I'm not really sure what you are arguing. Of course if its a one man job (or a couple) and the .. product or whatever isn't system/safety critical even having a PM is probably overkill and the overhead of "any" process likely just wrecks the output. But if you need to organize 50+ engineers and actually produce a ... product? That people/agencies/customers expect things of you sure as shit is going to need a definable goal, verifiable output and a somewhat chronological way of achieveing that. Waterfall it is (with reasonable iterations).
      - bratbag 3y ago
        Good luck building a chronology for 50+ engineers ahead of time that isn't either horrifically over estimated or instantly wrong. Waterfall is a pacifier for people who would rather have lies than uncertainty.
        mrmlz 3y ago
        Sure time estimation in developmentprojects is hard (impossible?), but thats hard in both waterfall and agile or whatever. But the often used CER -> PDR -> CDR -> TRR etc. is damn useful.
        datadrivenangel 3y ago
        >Waterfall is a pacifier for people who would rather have lies than uncertainty. This is the truth. It's painful to see, but some people would prefer to light millions of dollars on fire than admit they don't know.
        psunavy03 3y ago
        It's not just about "admitting they don't know." It's about giving up the control that they've used to define their entire professional lives as "managers." Thank you Frederick Winslow Taylor for your outdated ideas.
- deleted 3y ago
  [deleted]
- bipson 3y ago
  I think here is the common misunderstanding: No one says Agile can't have full blown requirements or the respective documents. To the contrary, you can't expect your dev to understand what he has to write without giving him a goal he is working on - Agile won't fix that. Agile just means you build in small steps from what you have and review your steps ASAP, instead of working multiple months based on some detailed documents, derived from other documents, written by some other department months ago, building stuff no one wanted in the first place.
  - sethammons 3y ago
    It seems the majority of comments miss what you are saying. We are an Agile Shop. We design things. We have to at scale. We review schema designs, query patterns, error handling strategies, monitoring, etc. Without that review, you get people doing silly things because most devs have less than 5 years experience. Deliver a minimal viable product and use customer feedback to iterate. The goal is to provide customer value as quickly and regularly as possible. This can include all the Excellence in Engineering standards you want.
black_13 3y ago
[dead]
jongjong 3y ago
People tend to go too far in one direction and then later over-correct in the opposite direction. Waterfall model is too much, scrum is too much. The optimal approach is almost always in-between and yet, frustratingly, nobody advocates for balance and nuance. Software development requires some planing and also some flexibility. One decade, companies think software architecture is the most important technical role in the universe, the next decade, the role barely even exists... What we need more of is common sense.
- mrmlz 3y ago
  Honestly I've never seen a waterfall model implemented without iterations. And you can always update the top-spec at later date if needed its just more difficult.. Which it should be.
  - SideburnsOfDoom 3y ago
    > Honestly I've never seen a waterfall model implemented without iterations. I have. It didn't reach the end.
    - mrmlz 3y ago
      Yeah - I mean, when I wrote the comment I ignored all the absolute disaster-projects I've seen. But those where doomed on so many levels I don't even wanna blame waterfall or agile or the in-betweens.
      - SideburnsOfDoom 3y ago
        So, "if you ignore the ones without iterations, I've never seen a waterfall model implemented without iterations."
        mrmlz 3y ago
        Hah yeah i guess you can interpret it like that, but i give the same leeway to agile when they skip the retros and the planning before the sprints etc. You cant blame the process if you dont even make an attempt of adhering to it :)
JackOfCrows 3y ago
For people who worship capitalism as much as folks here do, a lot of you sure don't get business processes. Yes, you wind up having to plan things, because the rest of the business world runs on deadlines and contracts and milestones and deliverables. Agile is, God forbid, almost like being an artist or low, disgusted whisper liberal arts major...when done correctly, in that you're iterating and tinkering and shipping and ideally you eventually get to good software sometime. Only the business world runs on quarterly results and shareholder value and being able to say "yes, we are shipping 10 great new features next quarter, we have a $20 million marketing campaign, new user acquisition campaign, outreach to lapsed users, and PR campaign all ready to go" gets you bonuses and increased stock prices and new investment rounds. Saying a bunch of Agile gobbledygook gets you...a job as a product manager or lead but you still gotta hit those milestones
- SideburnsOfDoom 3y ago
  The question that you're missing is: Which works better? Which delivers more better software, more reliably? Granted, the business world does run on quarterly results and shareholder value and being able to say ..., but you can say any damn thing that you like, only promising it doesn't guarantee that it will happen. Software development is always uncertain, the question is how best to tame that - ignore it or lean into it. All things being equal, of course the the business world would prefer the most detailed predictable plan. But, they just aren't equal. Software development benefits greatly from short feedback loops. In all phases - from debugging to finding out what the users really need. That is antithetical to big upfront plans.
  - goodlinks 3y ago
    There needs to be a stable narrative to shareholders, anything within your control should be managed. Moving all software development into some kind of skunkworks part of the business with 0 expected returns and only ever discussing roll outs may be one way to do it. (Shrug) Its why i think innovation is maybe a bit easier at a privately held or government owned org. They can look at the value more directly rather than through share value impact. This response is just about how big a deal the shareholder perspectove is, if you can fly under that radar then you stillneed to convince everyone else that you can be trusted to deliver eventually.
choeger 3y ago
Naturally everyone does waterfall, just at different points. Even if I did the leanest form of Kanban, I have to check the requirements of each ticket, understand them, translate them to maybe a prototype/discovery phase, come up with tests and an implementation strategy and verify the result somehow. That's just waterfall at the finest possible granularity. Waterfall doesn't mean not to iterate. It also doesn't mean to not build prototypes or MVPs. It just pushes the risky decisions back to people that are actually paid to make these decisions. Scrum is not just a framework for micromanaging, it also conveniently removes any domain responsibility from management. It's always "the team decides". (Of course, just on paper. Try to decide to do tech debt only for a couple of sprints with your team and see how management reacts.) Besides, waterfall is just the straw man of scrum zealots. I once proposed a new team structure where frontend and backend wouldn't be mixed because I observed that most stories didn't have an overlap but where rather mostly one or the other. The scrum master was shocked that I wanted to get rid of "cross functional" teams: "What do you want to do instead, waterfall?". For them it was inconceivable that cross functionality was not equivalent to scrum.
- hnben 3y ago
  > Waterfall doesn't mean not to iterate I don't want to be pedantic, but waterfall does mean exactly that by definition (or at least it used to). There are plenty of variations, that include prototypes or MVPs or iterations (e.g. V-Model, Spiral model). > waterfall is just the straw man of scrum zealots You can't just change the definition of a word, and then call people using the original definition "zealots", see https://en.wikipedia.org/wiki/Motte-and-bailey_fallacy https://en.wikipedia.org/wiki/Motte-and-bailey_fallacy
  - rramadass 3y ago
    > but waterfall does mean exactly that by definition No. This article The Myth of the 'Waterfall' SDLC gives you the details (archive version since the site seems to be down) - https://archive.is/jbBta https://archive.is/jbBta
  - atoav 3y ago
    The question is whether any of the "classical" examples for waterfall project managements out there didn't iterate at all. Having iterations is not incompatible with waterfall per se. E.g. you can plan to have 3 prototypes and then one final product beforehand. You can also decide to skip project phases if they are not needed. When people hear waterfall they think of a pure methodology, when in practise the methodology nearly always will be (and was) sacrificed if the need arises. So as we are discussing a thing we do in practise, we shouldn't discuss it as a theoretical framework in the vacuum of space, but as something that will be (a)bused in reality. As a film guy I am a big fan of clear up-front planing (you probably know some of the things that go into films, treatments, scripts, story-boards, concept art, ...) A film is very waterfall-y in how it is produced. But good directors maintain their flexibility on set by maintaining a balance in how serious they take the script on set and where other solutions present themselves that are a thousand times better, e.g. because the location/the actors or whatever is more striking than any writer could have imagined etc.
tcmb 3y ago
"The Internet brought agile, what will remote work bring?" Interesting question. What are the companies doing that have been working remotely already years before the pandemic? How is e.g. Gitlab working? Would love to hear people's experience. It's not like Agile cannot work with remote collaboration, just that waterfall is more suitable if people have a tendency to isolate and form silos.
- MyneOutside 3y ago
  Working remote for 10+ years and as a manager of software developers and developing myself, not sure I would see working remote or in house makes any difference on what processes you use. The processes you use, or try to use, are limited by the culture of management that work above and around you in other departments. Changing that culture to be accepting of the nature of software development is difficult, particularly in my case where we are interacting with non technically-minded people who are prone to make decisions based on their emotions and/or their own motivations regarding office politics.
- esafak 3y ago
  Initially it will bring more asynchronicity. More writing and recording. Later it might involve more XR.
marcus_holmes 3y ago
I disagree. Everyone does iterative development, always, but it comes disguised as waterfall/scrumfall/kanban/whatever. The product is developed using waterfall and then it gets put in front of a stakeholder, who has changed their mind in the time between the initial spec and the delivery. So another, smaller, waterfall is created. This is iterative development. And it always happens, because no-one gets product development right first time. We'd actually benefit from realising this. Which was the point of Agile - that you can change course during development and so develop the right thing through iteration. But Agile is (by definition) unpredictable. So you end up with this half-arsed solution that pretends to be predictable (except what it predicts is only the first iteration) and pretends to be Agile (except changing course is impossible) as the article says.
- The_Colonel 3y ago
  I think this ignores two major constraints - the "waterfall mindset" was about stakeholders (customers) believing they can get a finished product in a fixed timeline for a fixed price. What happened then was that after the delivery, the stakeholder was wildly unhappy with the result and demanded re-work, but that would be unplanned and un-budgeted. Many ugly arguments and finger pointing would ensue, which could repeat in several "iterations". The other thing which doesn't align with agile was that the stakeholder usually didn't participate in the development. They weren't reviewing the progress, they were typically waiting for the finished product to be delivered.
  - marcus_holmes 3y ago
    This is what I mean. Everyone does iterative, it's just that a lot of people don't realise it's iterative so do it badly.
- atoav 3y ago
  Two things: 1) You can end at the first iteration and call it a day. The only reason nobody does that is because they don't benefit from an unhappy customer 2) If you have experience, nothing stops you from putting some iterations into the waterfall. As you said the need for feedback rounds should be no surprise. Traditionally the downside to waterfall was that the distance between those iterations was too big. But smaller iterations and partly prototypes is nothing that wouldn't be allowed in waterfall if you'd like to do it. And if you find a PM that doesn't do the reasonable thing "because the methodology won't allow it" they are probably a bad PM.
rstuart4133 3y ago
The Zen of waterfall: At some level everything is waterfall. If anything is to get done at all the programmer has to make a plan, then put his head down, arse up and implement it. At another level nothing is waterfall. If the plan is successful it was shipped to the consumer (which may be the programmer himself), and it's very presence changes things for the consumer in ways that they didn't foresee. They then realise they need a new set of changes. What we call waterfall model is really referring to the scale. If the plan is grand, the specifications for such a grand plan need to be detailed, the implementation long, and the time between putting the head down and evaluating results is large, and we call it waterfall. But if the strategy is to explore the solution organically, the re-evaluations are frequent, the waterfall periods are short and we call the strategy something else. So in the end everything uses waterfall. The programmer would not get the long periods of intense focus he need to be productive without it. But also nothing is waterfall, because no plan can foresee everything, it must be continually re-evaluated in the light of unexpected changes it brings as it is implemented. From https://news.ycombinator.com/item?id=23970156 https://news.ycombinator.com/item?id=23970156 : by rstuart4133 on July 27, 2020
- btbuildem 3y ago
  If you want to get more abstract with it: waterfall is in two dimensions. The process of starting something and working toward completion can be represented as a waterfall (in 2D) but always and by definition must be represented as a line (in 1D) between two points: start and finish. If you project from 1D to 2D, that linearity will always be there (for completed projects, at least). That extra dimension will introduce additional flavour, and waterfall seems to be a very frequent one.
- mycall 3y ago
  This is known as iterative waterfall.
docflabby 3y ago
I think problem is product managers are very common and probably unavoidable in most organisations. Can't have product owners without actual ownership - without real equity in the product being developed do they really own anything?
- deleted 3y ago
  [deleted]
28304283409234 3y ago
Agile always was the answer to that nagging project manager asking “When will it be done?” By engaging with an answer the engineers kept being part of the discussion. Instead, they should have said: you don’t get to ask that question. That question has no place here. Engage with us in building the product and let us create and enjoy! Of course, the real world needs deadlines, timelines, roadmaps. And thus, agile is pointless. As it is just waterfall. In a different colour.
- ranguna 3y ago
  And then the project manager fires everyone and hires a team that gives them the answers they want. Even though most of the time the answers were wrong, but by that time the project manager is already neck deep with dependencies around the new team. That's why we can't have nice things. You depend on having nice project managers, which is rarely the case.
- switch007 3y ago
  Yup. Every single organisation I’ve worked at just paid lip service to the whole “you need to be involved in the process” observation. Unfortunately in my career I’ve only ever seen execs recede further and further away, hiring more and more PMs to replace them in that task. The self aggrandisement is getting worse and worse. They go away for retreat camps, and strategy meet-ups etc and pat themselves on the back about doing Very Important Work for weeks after each event Here it’s a class thing. They see themselves aligned with the capitalist owners more than the “factory workers” (the devs)
- esafak 3y ago
  Managers should be predicting and controlling the delivery time by clearing organizational roadblocks and shaping the work to be done, not how it is done.
timwaagh 3y ago
RUP really isn't waterfall anymore. It's not as linear. It's it's own thing, a sort of hybrid model. The nineties already knew the issue with expecting to get everything right the first time. The problem with it is it's too complicated. Your management with the mental capacity of a five year old just wouldn't understand it.
- rapnie 3y ago
  And RUP had iterations as well, and parallel disciplines. It had a lot of good to the overall idea. Are there modern variants? As a pattern library maybe, take just what you need. https://web.archive.org/web/20230322153353/https://sceweb.uhcl.edu/helm/RationalUnifiedProcess/ https://web.archive.org/web/20230322153353/https://sceweb.uh...
boffinAudio 3y ago
Waterfall is natural law. All technology goes through the process: Analysis - Requirements - Specification - Design - Implementation - Qualification - Testing - Release. You literally cannot have good technology without enforcing this workflow and properly completing each step. Its a function of the nature of the way the mind works to solve a problem and encode that solution into a workable technology. This natural law is resisted by those who do not have experience with it, and who also attempt to make their mark on the universe by either re-inventing this natural law, and slapping their own brand on it, or attempting to ignore key parts they don't like, and in which they are not competent, refactoring it over time only to discover .. as it is a natural law - that they end up re-inventing it, as well. And then slap their brand on it. But I can say with great confidence after 40 years of developing technology, that the more you resist this natural law, the worse your product will be. All failed technology can be evaluated on the basis of how well this workflow was manifested by those involved. Scrum is just New-Age Waterfall. Agile is just 'officious scrum'. 10x developers are merely individuals who have stopped resisting this natural law, and in so doing have worked out how to apply it productively in almost everything they do .. Another factor is that Waterfall is iterative. You can do it in grandiose fashion, and you can do it on a granular, microcosm-level. The more you iterate, the more water flows, and the better things get - but people tend to resist revising themselves. The inclination to check oneself before one wrecks oneself is either bred out, culturally, or reinforced through the gauntlet of fire that is the modern industry. The better you get at iterating over the waterfall, the higher quality your product.
- psd1 3y ago
  I'm glad you brought up iteration in the context of waterfall. I do agree that the elements of analysis/reqs/spec etc are all vital. What isn't obviously necessary is that they happen in a rigid linear sequence. The point of MVP is to get from implementation back to analysis & requirements. By iterating through the steps in a non-linear way, you get better results for each step. I'd argue that requirements gathering is the hardest to get right and the part that most frequently fails, probably because workers are not consulted and, even if they are, they don't speak the same language as the analysts. So I believe that waterfall is still a good methodology, but (in most cases) only if it is done as a directed cyclic graph rather than a strict linear sequence, such that blocks of steps can be repeated. This breaks estimation, but estimation is broken anyway. It offers that each step is, finally, done right, but recognising that the first attempt at each step is bound to be wrong and must be iterated on. Does this match your view?
jillesvangurp 3y ago
The exception to this is of course open source projects operating at a much larger scale than most software companies. But they have come up with highly structured ways to develop that don't involve planning out everything in detail, which is a waterfall thing to do. Linux project: releases regularly every eight weeks or so. Chrome & Firefox: same thing. In fact most well run larger OSS projects seem to end up using date driven milestones rather than content driven milestones. Inside these projects you of course find people or groups of people that may have their own roadmaps and plans but they keep these separate from the overall plan, which is simply to ship whatever on date x. Either their stuff is ready and it gets merged and released or it stays on a branch until it is ready for a next release. Branches changed everything for open source developers. Every change is now a mini project. The last step of that project is to merge it, when the project is over. Some branches only last minutes, some last months/years. And those can break down into sub branches/projects. That's actually how the Linux project moves forward. Mostly Linus Torvalds is continuously integrating what is deemed ready. The mistake many companies make over and over again is managing whole products as one project. It doesn't work. Stop doing that and life gets better. Look at how big companies release big software packages these days. They use calendar driven milestones as well. They have lots of internal projects about things that may or may not end up in there. But the releases mostly go out as planned, i.e. when they are supposed to go out with whatever is in a stable state. The show must go on. That wouldn't work with waterfall. They'd constantly be missing those deadlines. The only reasons for moving deadlines are usually quality related. I always like this notion of inversion of control. You don't plan what ships, just when you ship and at what level of quality. This is much easier to plan for. Basically it becomes a game of quality control. If it's not ready, don't merge it and it's not on the critical path to anything. The rest just involves looking at the calendar. Product owners hate this inversion of control because they want to put content on the calendar. A good PO actually understands this and can adapt to that. An inexperienced one will indeed revert to planning lots of silly content milestones in a calendar. And then you are doing waterfall and falling behind the unrealistic schedule. Simple solution: don't do that. A POs job is to prioritize and keep amount of work in progress low to ensure that the right things get done well and as soon as possible. A high amount of work in progress signals slipping deadlines, poorly prioritized work, and lots of people stressing out about everything being super important. Agile processes tried to fix this. But you still end up with POs in control and sweet talking to senior management. So you end up doing waterfall if they don't know what they are doing (which happens a lot). Writing this as a CTO who is doubling as the PO (until we can hire someone that is better at that than me, I'm not bored). My approach to this is very simple. I have a hundred things I want to get done. I pick the top two and do those and than we do some more. The hard part is picking the right two things. Not coming up with a hundred more things to do. That requires discipline and a grasp of business and tech. That's my job. A key insight here is that you shouldn't plan out those hundred things in any amount of detail until you are ready to do them. That's waterfall thinking. You are just building an inventory of crap that will never get done and is probably the wrong thing to do by the time you get around to doing those things months/years later. Instead, I set high level mid term goals and plan short term according to those goals. The short term is about what we do right now and making sure that is the best thing to do and then doing that and only that well. We have continuous deployment so I don't care about calendars. Sprints are our short term planning horizon. Day to day work is Kanban driven. Do the most important thing, do nothing else.
chadcmulligan 3y ago
I'm doing scrum and waterfall at the moment, customers are saying why is it costing so much? Managers are saying we're agile :-D what fun. We have design docs and requirements docs and epics and scrum masters and devops etc etc
- CuriousSkeptic 3y ago
  This is the kind if project where agile can truly shine. “Hey, customer, looks like we burnt through half your budget and it doesn’t look like we’ll be able to complete the full spec on what remains… I could ask you to increase the budget, but I think I have a better suggestion. You tell me which feature is most important to get in and well focus on that exclusively, and iterate on that until you're either happy with what you’ve got, or the budget runs out. Then lets take a decision on how to proceed from there if still needed”
  - alphager 3y ago
    Look, I paid you your asking price for the /full/ spec; now you're telling me you will only deliver it partially and have the gall to say I get to decide how much money on top I pay you after the deadline for the full spec has passed? Why on earth would I accept this?
    - abwizz 3y ago
      Look, we already catered to your "corrections" of the full spec here, here, there and then. surely this is work on our side that comes on top of the work we agreed upon. thing is, we already built the x, y and z features that your current solution is lacking, which already saves you top dollar on your business workflows. that's how good we are.
    - marcosdumay 3y ago
      Yeah, if the customer insists on waterfall, the customer will get waterfall. You can choose to accept this or leave the customer to yell at and sue somebody else.
BLKNSLVR 3y ago
Purely my anecdotal experience: Everything tends to waterfall because the natural human "thing" is to create a process which can be repeated to ensure success / quality / <thing deemed good>. If Agile thing worked, let's package it up into something repeatable and follow that to the letter... Oh, waterfall. Agile requires constantly working against that, which is hard, because Managers want and love safe and predictable and measurable against something concrete. Make a framework rather than a process, but that's a fine line. Agile ultimately needs trust in the team to be able to think on their feet for every scenario coming into their board. I'm my experience this trust either didn't exist, or is treated as a second class citizen to said safe, predicable, measurable, repeatable process. And I don't believe necessarily that Agile is the be all and end all.
shalmanese 3y ago
Waterfall is the default resting state of software development same as how _v3_final_FINAL_newedits_Finalv4_revised.doc is the default resting state of version control. It takes active energy to move away from that state and things inevitably drift back towards it without continued application of discipline.
solatic 3y ago
At its core, the true underlying problem is that you can't actually fully control another person's output while still holding onto any genuine agility. The design is going to leave out some Product features, the code is going to diverge slightly from the design, QA will find some things the code didn't account for, DevOps, Security, and Legal will each chime in with other things that weren't considered. If the organization requires that everyone is happy, then everyone gets to review the work at every stage (in order to express unhappiness and get it fixed before proceeding). The inevitable result is a Waterfall process to gatekeep how work moves to each stage. The inevitable result is a lack of agility. But if the organization can lean into the Truth that Nothing Is Perfect, that your product is not perfect, your design is not perfect, your code is not perfect, your scaling, observability, security, and compliance are not perfect, then suddenly, nobody needs anybody's approval. You ship, and you ship quickly. Then you focus on learning and iterative improvement. Make things better over time, but not necessarily right now. Getting rid of Waterfall is learning to let go and to live with imperfection.
- JohnFen 3y ago
  > Getting rid of Waterfall is learning to let go and to live with imperfection. To a degree. But I'm pretty sure one of the reasons why software quality is so much lower these days is because many teams take the idea too far.
- nicbou 3y ago
  The problem is that some things must be perfect, because they expose the company to significant risk. You can iterate on a feature but not on compliance.
  - ilrwbwrkhv 3y ago
    You can iterate on compliance too. Just ask Uber.
  - smugglerFlynn 3y ago
    You can iterate on compliance _implementation_ if you keep some tollgate that guarantees all compliance business requirements are eventually met. In my opinion this is not the same as waterfall, and leaves room for experiments and iterative learning. The (very) tricky part, in my experience, is to have these requirements captured in implementation-agnostic way. Even strict accounting reqs require huge amount of work for someone to deconstruct them down to actual compliance requirement, instead of something solution-specific like "we need to store X in component Y". Not everyone wants to invest into this just for agility sake.
baz00 3y ago
No we don't. Waterfall is probably the only way of managing large quantities of differently skilled people on a software project and actually getting somewhere though. True agile is for very small teams of highly competent people only, which are rare. There are some corrupt organisations out there selling process models which tell you you're getting the latter but actually sell you waterfall again with bits renamed. This drives people away who are sold the agile model at an interview because the people who are invested in it in the organisation don't know either process well enough to rationalise it. So people arrive, realise it's a shit show and leave again quickly. That's reality and it sucks. I think people need to talk more around the social and political structures of organisations that drive the process model rather than two camps because there's infinite varieties of both ways of doing things.
- marcosdumay 3y ago
  Waterfall always fails, on every kind of project management. It's a very recent invention, from the late 20th century, and mostly only "used" in theoretical works until a ridiculous small number of decades ago. It actually being used on practice is very likely the main reason why large projects cost so much nowadays. It is really not "the only way of managing large number of skilled people".
ferd 3y ago
Nitpick: The RUP is not waterfall. First sentence on Wikipedia: "The rational unified process is an iterative software development process framework created by the Rational Software Corporation, a division of IBM since 2003". Now, it IS very heavy and bureaucratic, and 99% of corporations applied it in the worst way possible, with heavy tooling and in strict waterfall fashion (even IBM/Rational consultants).
- psunavy03 3y ago
  Sort of like SAFe, which . . . oh, wait! Was developed by the exact same people once they decided to get on the Agile gravy train! And I say that as someone who works in Agile, supports it 100%, and has no time for the "it's a scam" whining.
hirvi74 3y ago
Perhaps not "true" Agile, but I strongly believe modern, corporate Agile with Scrum with nothing more than a micromanagement framework.
- datadrivenangel 3y ago
  Micromanagement, it's the only way to safely scale agile and ensure you're getting maximum resource utilization.
Ozzie_osman 3y ago
Every project needs some time where you decide what to build, and some time where you build it. The idea of agile and other methods wasn't to eliminate that because you can't. The idea was to reduce the chunk size for each batch (ie flesh out a part of the feature instead of the full product), reduce the detail needed in each chunk, and hence create faster cycle times and more room for feedbaxk, which increases learning, improves speed, introduces autonomy, and decreases risk. "Waterfall" in the broader sense is inevitable. What we moved away from was "let's spend 6 months writing up a 300 page specifications doc for this new Operating System that will be handed off to 100 engs" to "a PM and designer can mockup this feature in a few days with some input from the person who will implement it, then the person who will implement it will build it with input from them along the way".
yafbum 3y ago
No amount methodology is going to make gold come out of a team that doesn't deeply understand the reasons for building what they're building. I work with a small-a agile team, and the devs who "get it" just make it happen without a waterfall. The ones who don't get it get a super detailed spec to minimize the chances they'll mess it up, and still mess it up. The ones in the middle know when to reach out to ask for an out-of-cycle product clarification before building a useless pile of code. The main difference is, do they understand why they're building what they're building? Or do they show up, punch through a bunch of tickets, go home? Really hard to fix that with process when that's not working. It starts with good hiring, building a vision together for what we're building, and communicating it repeatedly.
- thoughtpalette 3y ago
  I think this pretty much sums it up. If you have a skilled team with deep domain knowledge, you can afford to not be as detailed in the planning process as the team is most likely already aligned. If you have a skilled team without deep domain knowledge, you need that sense of unity and alignment, that usually comes through more detailed planning. It's not one's correct or one's wrong, it just depends on the team composition.
- loa_in_ 3y ago
  Assuming a team of all at least proficient developers, one who's living a peaceful life outside of work is going to be put into the first category, and people with more chaos in their lives (not by their choice) will fall into the latter category. As their life situation changes they have a bigger chance to lose context, forget who's who or miss out on word of mouth; or conversely, become a sponge for company culture and leave the office with energy for fulfilling projects outside of work and get involved with, for example, open source where they gain insight that they take into work later, for example perspectives on business specific knowledge or some analogues.
  - nophunphil 3y ago
    I think some developers are just not proficient though. You make a fair point about life circumstances, but not everyone can be prescribed to this track.
teddyh 3y ago
It’s like how capitalism always end up with monopolies, absent any counteracting force.
sjducb 3y ago
If you’ve got a release deadline, and a list of features that need to be ready by the deadline then you’ll end up doing waterfall. If you have flexibility about when features can be released then you can do agile. I wish the agile and scrum evangelists were honest about this. The most agile company I ever worked for never talked about scrum or agile. We just asked customers what they wanted and integrated it into the product. When features were ready we released them. There was no roadmap or timetable. Sometimes customers would ask us when things will be ready, we always pushed back on giving dates.
datadrivenangel 3y ago
There is an inherent tension between planning and iteration. As you plan for more and more unknowns, the cost of planning increases towards infinity. There is an optimal amount of planning, but knowing it exactly in advance is suboptimal in terms of planning planning cost.
cynusx 3y ago
You know, waterfall only fails when you build things nobody wants. If you, on the contrary, invest ridiculous amounts of time in user research and only build things that are validated then waterfall is fine.
fungiblecog 3y ago
All of these methodologies fall down because the incentives of the people involved rarely align. the more people involved the worse it gets
AndrewKemendo 3y ago
My favorite part about being in software over the last 20 years is watching software engineers reinvent all of the processes that every other engineering discipline has been utilizing for tens if not hundreds of years from first principles. I suppose it makes sense given how young SWE is, but I think the fact that cybernetics died and computer science was treated more like mathematics than engineering, so software people rarely if ever practice system thinking. It’s also striking that a software engineering undergraduate will never take a course on ethics, or social theory, as part of a core engineering curriculum. Whereas within the mechanical and specifically civil engineering, you will absolutely take courses that show you your social impact on your engineering. That is just never happening in software pedagogy

152 points by KingOfCoders 3y ago

deleted 3y ago

[deleted]

Easy, as somehow discussed on the article. Beyond the IT department everyone else has very to low incentives to adopt "those crazy ideas from computer guys", they care about deadlines, roadmaps, when marketing campaigns can be organised, with what content, what should be on the contracts with business partners,... Ever since we got those ideas around 2000, the best outcome if agile is adopted at all, is scrumfall as the article describes. However I have also seen many companies go back to classical requirements documents, and all related processes, as if agile had never been in the building.

mrmlz 3y ago
Yeah, because waterfall with iterations is better than SCRUM/AGILE/WHATEVER. God intended water to run from the TOP to the BOTTOM. Don't fight gravity.
- psunavy03 3y ago
  [flagged]
- atoav 3y ago
  The best results I have ever seen have been projects that people on the bottom did themselves, because they needed it without the people on the top ever knowing about it. The worst results I have ever seen have been projects where the people on the bottom stopped caring because the people on the top tried to make them ignore the laws of nature. Also: God is dead
  - mrmlz 3y ago
    I'm not really sure what you are arguing. Of course if its a one man job (or a couple) and the .. product or whatever isn't system/safety critical even having a PM is probably overkill and the overhead of "any" process likely just wrecks the output. But if you need to organize 50+ engineers and actually produce a ... product? That people/agencies/customers expect things of you sure as shit is going to need a definable goal, verifiable output and a somewhat chronological way of achieveing that. Waterfall it is (with reasonable iterations).
    - bratbag 3y ago
      Good luck building a chronology for 50+ engineers ahead of time that isn't either horrifically over estimated or instantly wrong. Waterfall is a pacifier for people who would rather have lies than uncertainty.
      - mrmlz 3y ago
        Sure time estimation in developmentprojects is hard (impossible?), but thats hard in both waterfall and agile or whatever. But the often used CER -> PDR -> CDR -> TRR etc. is damn useful.
      - datadrivenangel 3y ago
        >Waterfall is a pacifier for people who would rather have lies than uncertainty. This is the truth. It's painful to see, but some people would prefer to light millions of dollars on fire than admit they don't know.
        psunavy03 3y ago
        It's not just about "admitting they don't know." It's about giving up the control that they've used to define their entire professional lives as "managers." Thank you Frederick Winslow Taylor for your outdated ideas.
deleted 3y ago
[deleted]
bipson 3y ago
I think here is the common misunderstanding: No one says Agile can't have full blown requirements or the respective documents. To the contrary, you can't expect your dev to understand what he has to write without giving him a goal he is working on - Agile won't fix that. Agile just means you build in small steps from what you have and review your steps ASAP, instead of working multiple months based on some detailed documents, derived from other documents, written by some other department months ago, building stuff no one wanted in the first place.
- sethammons 3y ago
  It seems the majority of comments miss what you are saying. We are an Agile Shop. We design things. We have to at scale. We review schema designs, query patterns, error handling strategies, monitoring, etc. Without that review, you get people doing silly things because most devs have less than 5 years experience. Deliver a minimal viable product and use customer feedback to iterate. The goal is to provide customer value as quickly and regularly as possible. This can include all the Excellence in Engineering standards you want.

black_13 3y ago

[dead]

jongjong 3y ago

People tend to go too far in one direction and then later over-correct in the opposite direction. Waterfall model is too much, scrum is too much. The optimal approach is almost always in-between and yet, frustratingly, nobody advocates for balance and nuance. Software development requires some planing and also some flexibility. One decade, companies think software architecture is the most important technical role in the universe, the next decade, the role barely even exists... What we need more of is common sense.

mrmlz 3y ago
Honestly I've never seen a waterfall model implemented without iterations. And you can always update the top-spec at later date if needed its just more difficult.. Which it should be.
- SideburnsOfDoom 3y ago
  > Honestly I've never seen a waterfall model implemented without iterations. I have. It didn't reach the end.
  - mrmlz 3y ago
    Yeah - I mean, when I wrote the comment I ignored all the absolute disaster-projects I've seen. But those where doomed on so many levels I don't even wanna blame waterfall or agile or the in-betweens.
    - SideburnsOfDoom 3y ago
      So, "if you ignore the ones without iterations, I've never seen a waterfall model implemented without iterations."
      - mrmlz 3y ago
        Hah yeah i guess you can interpret it like that, but i give the same leeway to agile when they skip the retros and the planning before the sprints etc. You cant blame the process if you dont even make an attempt of adhering to it :)

JackOfCrows 3y ago

For people who worship capitalism as much as folks here do, a lot of you sure don't get business processes. Yes, you wind up having to plan things, because the rest of the business world runs on deadlines and contracts and milestones and deliverables. Agile is, God forbid, almost like being an artist or low, disgusted whisper liberal arts major...when done correctly, in that you're iterating and tinkering and shipping and ideally you eventually get to good software sometime. Only the business world runs on quarterly results and shareholder value and being able to say "yes, we are shipping 10 great new features next quarter, we have a $20 million marketing campaign, new user acquisition campaign, outreach to lapsed users, and PR campaign all ready to go" gets you bonuses and increased stock prices and new investment rounds. Saying a bunch of Agile gobbledygook gets you...a job as a product manager or lead but you still gotta hit those milestones

SideburnsOfDoom 3y ago
The question that you're missing is: Which works better? Which delivers more better software, more reliably? Granted, the business world does run on quarterly results and shareholder value and being able to say ..., but you can say any damn thing that you like, only promising it doesn't guarantee that it will happen. Software development is always uncertain, the question is how best to tame that - ignore it or lean into it. All things being equal, of course the the business world would prefer the most detailed predictable plan. But, they just aren't equal. Software development benefits greatly from short feedback loops. In all phases - from debugging to finding out what the users really need. That is antithetical to big upfront plans.
- goodlinks 3y ago
  There needs to be a stable narrative to shareholders, anything within your control should be managed. Moving all software development into some kind of skunkworks part of the business with 0 expected returns and only ever discussing roll outs may be one way to do it. (Shrug) Its why i think innovation is maybe a bit easier at a privately held or government owned org. They can look at the value more directly rather than through share value impact. This response is just about how big a deal the shareholder perspectove is, if you can fly under that radar then you stillneed to convince everyone else that you can be trusted to deliver eventually.

choeger 3y ago

Naturally everyone does waterfall, just at different points. Even if I did the leanest form of Kanban, I have to check the requirements of each ticket, understand them, translate them to maybe a prototype/discovery phase, come up with tests and an implementation strategy and verify the result somehow. That's just waterfall at the finest possible granularity. Waterfall doesn't mean not to iterate. It also doesn't mean to not build prototypes or MVPs. It just pushes the risky decisions back to people that are actually paid to make these decisions. Scrum is not just a framework for micromanaging, it also conveniently removes any domain responsibility from management. It's always "the team decides". (Of course, just on paper. Try to decide to do tech debt only for a couple of sprints with your team and see how management reacts.) Besides, waterfall is just the straw man of scrum zealots. I once proposed a new team structure where frontend and backend wouldn't be mixed because I observed that most stories didn't have an overlap but where rather mostly one or the other. The scrum master was shocked that I wanted to get rid of "cross functional" teams: "What do you want to do instead, waterfall?". For them it was inconceivable that cross functionality was not equivalent to scrum.

hnben 3y ago
> Waterfall doesn't mean not to iterate I don't want to be pedantic, but waterfall does mean exactly that by definition (or at least it used to). There are plenty of variations, that include prototypes or MVPs or iterations (e.g. V-Model, Spiral model). > waterfall is just the straw man of scrum zealots You can't just change the definition of a word, and then call people using the original definition "zealots", see https://en.wikipedia.org/wiki/Motte-and-bailey_fallacy https://en.wikipedia.org/wiki/Motte-and-bailey_fallacy
- rramadass 3y ago
  > but waterfall does mean exactly that by definition No. This article The Myth of the 'Waterfall' SDLC gives you the details (archive version since the site seems to be down) - https://archive.is/jbBta https://archive.is/jbBta
- atoav 3y ago
  The question is whether any of the "classical" examples for waterfall project managements out there didn't iterate at all. Having iterations is not incompatible with waterfall per se. E.g. you can plan to have 3 prototypes and then one final product beforehand. You can also decide to skip project phases if they are not needed. When people hear waterfall they think of a pure methodology, when in practise the methodology nearly always will be (and was) sacrificed if the need arises. So as we are discussing a thing we do in practise, we shouldn't discuss it as a theoretical framework in the vacuum of space, but as something that will be (a)bused in reality. As a film guy I am a big fan of clear up-front planing (you probably know some of the things that go into films, treatments, scripts, story-boards, concept art, ...) A film is very waterfall-y in how it is produced. But good directors maintain their flexibility on set by maintaining a balance in how serious they take the script on set and where other solutions present themselves that are a thousand times better, e.g. because the location/the actors or whatever is more striking than any writer could have imagined etc.

tcmb 3y ago

"The Internet brought agile, what will remote work bring?" Interesting question. What are the companies doing that have been working remotely already years before the pandemic? How is e.g. Gitlab working? Would love to hear people's experience. It's not like Agile cannot work with remote collaboration, just that waterfall is more suitable if people have a tendency to isolate and form silos.

MyneOutside 3y ago
Working remote for 10+ years and as a manager of software developers and developing myself, not sure I would see working remote or in house makes any difference on what processes you use. The processes you use, or try to use, are limited by the culture of management that work above and around you in other departments. Changing that culture to be accepting of the nature of software development is difficult, particularly in my case where we are interacting with non technically-minded people who are prone to make decisions based on their emotions and/or their own motivations regarding office politics.
esafak 3y ago
Initially it will bring more asynchronicity. More writing and recording. Later it might involve more XR.

marcus_holmes 3y ago

I disagree. Everyone does iterative development, always, but it comes disguised as waterfall/scrumfall/kanban/whatever. The product is developed using waterfall and then it gets put in front of a stakeholder, who has changed their mind in the time between the initial spec and the delivery. So another, smaller, waterfall is created. This is iterative development. And it always happens, because no-one gets product development right first time. We'd actually benefit from realising this. Which was the point of Agile - that you can change course during development and so develop the right thing through iteration. But Agile is (by definition) unpredictable. So you end up with this half-arsed solution that pretends to be predictable (except what it predicts is only the first iteration) and pretends to be Agile (except changing course is impossible) as the article says.

The_Colonel 3y ago
I think this ignores two major constraints - the "waterfall mindset" was about stakeholders (customers) believing they can get a finished product in a fixed timeline for a fixed price. What happened then was that after the delivery, the stakeholder was wildly unhappy with the result and demanded re-work, but that would be unplanned and un-budgeted. Many ugly arguments and finger pointing would ensue, which could repeat in several "iterations". The other thing which doesn't align with agile was that the stakeholder usually didn't participate in the development. They weren't reviewing the progress, they were typically waiting for the finished product to be delivered.
- marcus_holmes 3y ago
  This is what I mean. Everyone does iterative, it's just that a lot of people don't realise it's iterative so do it badly.
atoav 3y ago
Two things: 1) You can end at the first iteration and call it a day. The only reason nobody does that is because they don't benefit from an unhappy customer 2) If you have experience, nothing stops you from putting some iterations into the waterfall. As you said the need for feedback rounds should be no surprise. Traditionally the downside to waterfall was that the distance between those iterations was too big. But smaller iterations and partly prototypes is nothing that wouldn't be allowed in waterfall if you'd like to do it. And if you find a PM that doesn't do the reasonable thing "because the methodology won't allow it" they are probably a bad PM.

rstuart4133 3y ago

The Zen of waterfall: At some level everything is waterfall. If anything is to get done at all the programmer has to make a plan, then put his head down, arse up and implement it. At another level nothing is waterfall. If the plan is successful it was shipped to the consumer (which may be the programmer himself), and it's very presence changes things for the consumer in ways that they didn't foresee. They then realise they need a new set of changes. What we call waterfall model is really referring to the scale. If the plan is grand, the specifications for such a grand plan need to be detailed, the implementation long, and the time between putting the head down and evaluating results is large, and we call it waterfall. But if the strategy is to explore the solution organically, the re-evaluations are frequent, the waterfall periods are short and we call the strategy something else. So in the end everything uses waterfall. The programmer would not get the long periods of intense focus he need to be productive without it. But also nothing is waterfall, because no plan can foresee everything, it must be continually re-evaluated in the light of unexpected changes it brings as it is implemented. From https://news.ycombinator.com/item?id=23970156 https://news.ycombinator.com/item?id=23970156 : by rstuart4133 on July 27, 2020

btbuildem 3y ago
If you want to get more abstract with it: waterfall is in two dimensions. The process of starting something and working toward completion can be represented as a waterfall (in 2D) but always and by definition must be represented as a line (in 1D) between two points: start and finish. If you project from 1D to 2D, that linearity will always be there (for completed projects, at least). That extra dimension will introduce additional flavour, and waterfall seems to be a very frequent one.
mycall 3y ago
This is known as iterative waterfall.

docflabby 3y ago

I think problem is product managers are very common and probably unavoidable in most organisations. Can't have product owners without actual ownership - without real equity in the product being developed do they really own anything?

deleted 3y ago
[deleted]

28304283409234 3y ago

Agile always was the answer to that nagging project manager asking “When will it be done?” By engaging with an answer the engineers kept being part of the discussion. Instead, they should have said: you don’t get to ask that question. That question has no place here. Engage with us in building the product and let us create and enjoy! Of course, the real world needs deadlines, timelines, roadmaps. And thus, agile is pointless. As it is just waterfall. In a different colour.

ranguna 3y ago
And then the project manager fires everyone and hires a team that gives them the answers they want. Even though most of the time the answers were wrong, but by that time the project manager is already neck deep with dependencies around the new team. That's why we can't have nice things. You depend on having nice project managers, which is rarely the case.
switch007 3y ago
Yup. Every single organisation I’ve worked at just paid lip service to the whole “you need to be involved in the process” observation. Unfortunately in my career I’ve only ever seen execs recede further and further away, hiring more and more PMs to replace them in that task. The self aggrandisement is getting worse and worse. They go away for retreat camps, and strategy meet-ups etc and pat themselves on the back about doing Very Important Work for weeks after each event Here it’s a class thing. They see themselves aligned with the capitalist owners more than the “factory workers” (the devs)
esafak 3y ago
Managers should be predicting and controlling the delivery time by clearing organizational roadblocks and shaping the work to be done, not how it is done.

timwaagh 3y ago

RUP really isn't waterfall anymore. It's not as linear. It's it's own thing, a sort of hybrid model. The nineties already knew the issue with expecting to get everything right the first time. The problem with it is it's too complicated. Your management with the mental capacity of a five year old just wouldn't understand it.

rapnie 3y ago
And RUP had iterations as well, and parallel disciplines. It had a lot of good to the overall idea. Are there modern variants? As a pattern library maybe, take just what you need. https://web.archive.org/web/20230322153353/https://sceweb.uhcl.edu/helm/RationalUnifiedProcess/ https://web.archive.org/web/20230322153353/https://sceweb.uh...

boffinAudio 3y ago

Waterfall is natural law. All technology goes through the process: Analysis - Requirements - Specification - Design - Implementation - Qualification - Testing - Release. You literally cannot have good technology without enforcing this workflow and properly completing each step. Its a function of the nature of the way the mind works to solve a problem and encode that solution into a workable technology. This natural law is resisted by those who do not have experience with it, and who also attempt to make their mark on the universe by either re-inventing this natural law, and slapping their own brand on it, or attempting to ignore key parts they don't like, and in which they are not competent, refactoring it over time only to discover .. as it is a natural law - that they end up re-inventing it, as well. And then slap their brand on it. But I can say with great confidence after 40 years of developing technology, that the more you resist this natural law, the worse your product will be. All failed technology can be evaluated on the basis of how well this workflow was manifested by those involved. Scrum is just New-Age Waterfall. Agile is just 'officious scrum'. 10x developers are merely individuals who have stopped resisting this natural law, and in so doing have worked out how to apply it productively in almost everything they do .. Another factor is that Waterfall is iterative. You can do it in grandiose fashion, and you can do it on a granular, microcosm-level. The more you iterate, the more water flows, and the better things get - but people tend to resist revising themselves. The inclination to check oneself before one wrecks oneself is either bred out, culturally, or reinforced through the gauntlet of fire that is the modern industry. The better you get at iterating over the waterfall, the higher quality your product.

psd1 3y ago
I'm glad you brought up iteration in the context of waterfall. I do agree that the elements of analysis/reqs/spec etc are all vital. What isn't obviously necessary is that they happen in a rigid linear sequence. The point of MVP is to get from implementation back to analysis & requirements. By iterating through the steps in a non-linear way, you get better results for each step. I'd argue that requirements gathering is the hardest to get right and the part that most frequently fails, probably because workers are not consulted and, even if they are, they don't speak the same language as the analysts. So I believe that waterfall is still a good methodology, but (in most cases) only if it is done as a directed cyclic graph rather than a strict linear sequence, such that blocks of steps can be repeated. This breaks estimation, but estimation is broken anyway. It offers that each step is, finally, done right, but recognising that the first attempt at each step is bound to be wrong and must be iterated on. Does this match your view?

jillesvangurp 3y ago

The exception to this is of course open source projects operating at a much larger scale than most software companies. But they have come up with highly structured ways to develop that don't involve planning out everything in detail, which is a waterfall thing to do. Linux project: releases regularly every eight weeks or so. Chrome & Firefox: same thing. In fact most well run larger OSS projects seem to end up using date driven milestones rather than content driven milestones. Inside these projects you of course find people or groups of people that may have their own roadmaps and plans but they keep these separate from the overall plan, which is simply to ship whatever on date x. Either their stuff is ready and it gets merged and released or it stays on a branch until it is ready for a next release. Branches changed everything for open source developers. Every change is now a mini project. The last step of that project is to merge it, when the project is over. Some branches only last minutes, some last months/years. And those can break down into sub branches/projects. That's actually how the Linux project moves forward. Mostly Linus Torvalds is continuously integrating what is deemed ready. The mistake many companies make over and over again is managing whole products as one project. It doesn't work. Stop doing that and life gets better. Look at how big companies release big software packages these days. They use calendar driven milestones as well. They have lots of internal projects about things that may or may not end up in there. But the releases mostly go out as planned, i.e. when they are supposed to go out with whatever is in a stable state. The show must go on. That wouldn't work with waterfall. They'd constantly be missing those deadlines. The only reasons for moving deadlines are usually quality related. I always like this notion of inversion of control. You don't plan what ships, just when you ship and at what level of quality. This is much easier to plan for. Basically it becomes a game of quality control. If it's not ready, don't merge it and it's not on the critical path to anything. The rest just involves looking at the calendar. Product owners hate this inversion of control because they want to put content on the calendar. A good PO actually understands this and can adapt to that. An inexperienced one will indeed revert to planning lots of silly content milestones in a calendar. And then you are doing waterfall and falling behind the unrealistic schedule. Simple solution: don't do that. A POs job is to prioritize and keep amount of work in progress low to ensure that the right things get done well and as soon as possible. A high amount of work in progress signals slipping deadlines, poorly prioritized work, and lots of people stressing out about everything being super important. Agile processes tried to fix this. But you still end up with POs in control and sweet talking to senior management. So you end up doing waterfall if they don't know what they are doing (which happens a lot). Writing this as a CTO who is doubling as the PO (until we can hire someone that is better at that than me, I'm not bored). My approach to this is very simple. I have a hundred things I want to get done. I pick the top two and do those and than we do some more. The hard part is picking the right two things. Not coming up with a hundred more things to do. That requires discipline and a grasp of business and tech. That's my job. A key insight here is that you shouldn't plan out those hundred things in any amount of detail until you are ready to do them. That's waterfall thinking. You are just building an inventory of crap that will never get done and is probably the wrong thing to do by the time you get around to doing those things months/years later. Instead, I set high level mid term goals and plan short term according to those goals. The short term is about what we do right now and making sure that is the best thing to do and then doing that and only that well. We have continuous deployment so I don't care about calendars. Sprints are our short term planning horizon. Day to day work is Kanban driven. Do the most important thing, do nothing else.

chadcmulligan 3y ago

I'm doing scrum and waterfall at the moment, customers are saying why is it costing so much? Managers are saying we're agile :-D what fun. We have design docs and requirements docs and epics and scrum masters and devops etc etc

CuriousSkeptic 3y ago
This is the kind if project where agile can truly shine. “Hey, customer, looks like we burnt through half your budget and it doesn’t look like we’ll be able to complete the full spec on what remains… I could ask you to increase the budget, but I think I have a better suggestion. You tell me which feature is most important to get in and well focus on that exclusively, and iterate on that until you're either happy with what you’ve got, or the budget runs out. Then lets take a decision on how to proceed from there if still needed”
- alphager 3y ago
  Look, I paid you your asking price for the /full/ spec; now you're telling me you will only deliver it partially and have the gall to say I get to decide how much money on top I pay you after the deadline for the full spec has passed? Why on earth would I accept this?
  - abwizz 3y ago
    Look, we already catered to your "corrections" of the full spec here, here, there and then. surely this is work on our side that comes on top of the work we agreed upon. thing is, we already built the x, y and z features that your current solution is lacking, which already saves you top dollar on your business workflows. that's how good we are.
  - marcosdumay 3y ago
    Yeah, if the customer insists on waterfall, the customer will get waterfall. You can choose to accept this or leave the customer to yell at and sue somebody else.

BLKNSLVR 3y ago

Purely my anecdotal experience: Everything tends to waterfall because the natural human "thing" is to create a process which can be repeated to ensure success / quality / <thing deemed good>. If Agile thing worked, let's package it up into something repeatable and follow that to the letter... Oh, waterfall. Agile requires constantly working against that, which is hard, because Managers want and love safe and predictable and measurable against something concrete. Make a framework rather than a process, but that's a fine line. Agile ultimately needs trust in the team to be able to think on their feet for every scenario coming into their board. I'm my experience this trust either didn't exist, or is treated as a second class citizen to said safe, predicable, measurable, repeatable process. And I don't believe necessarily that Agile is the be all and end all.

shalmanese 3y ago

Waterfall is the default resting state of software development same as how _v3_final_FINAL_newedits_Finalv4_revised.doc is the default resting state of version control. It takes active energy to move away from that state and things inevitably drift back towards it without continued application of discipline.

solatic 3y ago

At its core, the true underlying problem is that you can't actually fully control another person's output while still holding onto any genuine agility. The design is going to leave out some Product features, the code is going to diverge slightly from the design, QA will find some things the code didn't account for, DevOps, Security, and Legal will each chime in with other things that weren't considered. If the organization requires that everyone is happy, then everyone gets to review the work at every stage (in order to express unhappiness and get it fixed before proceeding). The inevitable result is a Waterfall process to gatekeep how work moves to each stage. The inevitable result is a lack of agility. But if the organization can lean into the Truth that Nothing Is Perfect, that your product is not perfect, your design is not perfect, your code is not perfect, your scaling, observability, security, and compliance are not perfect, then suddenly, nobody needs anybody's approval. You ship, and you ship quickly. Then you focus on learning and iterative improvement. Make things better over time, but not necessarily right now. Getting rid of Waterfall is learning to let go and to live with imperfection.

JohnFen 3y ago
> Getting rid of Waterfall is learning to let go and to live with imperfection. To a degree. But I'm pretty sure one of the reasons why software quality is so much lower these days is because many teams take the idea too far.
nicbou 3y ago
The problem is that some things must be perfect, because they expose the company to significant risk. You can iterate on a feature but not on compliance.
- ilrwbwrkhv 3y ago
  You can iterate on compliance too. Just ask Uber.
- smugglerFlynn 3y ago
  You can iterate on compliance _implementation_ if you keep some tollgate that guarantees all compliance business requirements are eventually met. In my opinion this is not the same as waterfall, and leaves room for experiments and iterative learning. The (very) tricky part, in my experience, is to have these requirements captured in implementation-agnostic way. Even strict accounting reqs require huge amount of work for someone to deconstruct them down to actual compliance requirement, instead of something solution-specific like "we need to store X in component Y". Not everyone wants to invest into this just for agility sake.

baz00 3y ago

No we don't. Waterfall is probably the only way of managing large quantities of differently skilled people on a software project and actually getting somewhere though. True agile is for very small teams of highly competent people only, which are rare. There are some corrupt organisations out there selling process models which tell you you're getting the latter but actually sell you waterfall again with bits renamed. This drives people away who are sold the agile model at an interview because the people who are invested in it in the organisation don't know either process well enough to rationalise it. So people arrive, realise it's a shit show and leave again quickly. That's reality and it sucks. I think people need to talk more around the social and political structures of organisations that drive the process model rather than two camps because there's infinite varieties of both ways of doing things.

marcosdumay 3y ago
Waterfall always fails, on every kind of project management. It's a very recent invention, from the late 20th century, and mostly only "used" in theoretical works until a ridiculous small number of decades ago. It actually being used on practice is very likely the main reason why large projects cost so much nowadays. It is really not "the only way of managing large number of skilled people".

ferd 3y ago

Nitpick: The RUP is not waterfall. First sentence on Wikipedia: "The rational unified process is an iterative software development process framework created by the Rational Software Corporation, a division of IBM since 2003". Now, it IS very heavy and bureaucratic, and 99% of corporations applied it in the worst way possible, with heavy tooling and in strict waterfall fashion (even IBM/Rational consultants).

psunavy03 3y ago
Sort of like SAFe, which . . . oh, wait! Was developed by the exact same people once they decided to get on the Agile gravy train! And I say that as someone who works in Agile, supports it 100%, and has no time for the "it's a scam" whining.

hirvi74 3y ago

Perhaps not "true" Agile, but I strongly believe modern, corporate Agile with Scrum with nothing more than a micromanagement framework.

datadrivenangel 3y ago
Micromanagement, it's the only way to safely scale agile and ensure you're getting maximum resource utilization.

Ozzie_osman 3y ago

Every project needs some time where you decide what to build, and some time where you build it. The idea of agile and other methods wasn't to eliminate that because you can't. The idea was to reduce the chunk size for each batch (ie flesh out a part of the feature instead of the full product), reduce the detail needed in each chunk, and hence create faster cycle times and more room for feedbaxk, which increases learning, improves speed, introduces autonomy, and decreases risk. "Waterfall" in the broader sense is inevitable. What we moved away from was "let's spend 6 months writing up a 300 page specifications doc for this new Operating System that will be handed off to 100 engs" to "a PM and designer can mockup this feature in a few days with some input from the person who will implement it, then the person who will implement it will build it with input from them along the way".

yafbum 3y ago

No amount methodology is going to make gold come out of a team that doesn't deeply understand the reasons for building what they're building. I work with a small-a agile team, and the devs who "get it" just make it happen without a waterfall. The ones who don't get it get a super detailed spec to minimize the chances they'll mess it up, and still mess it up. The ones in the middle know when to reach out to ask for an out-of-cycle product clarification before building a useless pile of code. The main difference is, do they understand why they're building what they're building? Or do they show up, punch through a bunch of tickets, go home? Really hard to fix that with process when that's not working. It starts with good hiring, building a vision together for what we're building, and communicating it repeatedly.

thoughtpalette 3y ago
I think this pretty much sums it up. If you have a skilled team with deep domain knowledge, you can afford to not be as detailed in the planning process as the team is most likely already aligned. If you have a skilled team without deep domain knowledge, you need that sense of unity and alignment, that usually comes through more detailed planning. It's not one's correct or one's wrong, it just depends on the team composition.
loa_in_ 3y ago
Assuming a team of all at least proficient developers, one who's living a peaceful life outside of work is going to be put into the first category, and people with more chaos in their lives (not by their choice) will fall into the latter category. As their life situation changes they have a bigger chance to lose context, forget who's who or miss out on word of mouth; or conversely, become a sponge for company culture and leave the office with energy for fulfilling projects outside of work and get involved with, for example, open source where they gain insight that they take into work later, for example perspectives on business specific knowledge or some analogues.
- nophunphil 3y ago
  I think some developers are just not proficient though. You make a fair point about life circumstances, but not everyone can be prescribed to this track.

teddyh 3y ago

It’s like how capitalism always end up with monopolies, absent any counteracting force.

sjducb 3y ago

If you’ve got a release deadline, and a list of features that need to be ready by the deadline then you’ll end up doing waterfall. If you have flexibility about when features can be released then you can do agile. I wish the agile and scrum evangelists were honest about this. The most agile company I ever worked for never talked about scrum or agile. We just asked customers what they wanted and integrated it into the product. When features were ready we released them. There was no roadmap or timetable. Sometimes customers would ask us when things will be ready, we always pushed back on giving dates.

datadrivenangel 3y ago

There is an inherent tension between planning and iteration. As you plan for more and more unknowns, the cost of planning increases towards infinity. There is an optimal amount of planning, but knowing it exactly in advance is suboptimal in terms of planning planning cost.

cynusx 3y ago

You know, waterfall only fails when you build things nobody wants. If you, on the contrary, invest ridiculous amounts of time in user research and only build things that are validated then waterfall is fine.

fungiblecog 3y ago

All of these methodologies fall down because the incentives of the people involved rarely align. the more people involved the worse it gets

AndrewKemendo 3y ago

My favorite part about being in software over the last 20 years is watching software engineers reinvent all of the processes that every other engineering discipline has been utilizing for tens if not hundreds of years from first principles. I suppose it makes sense given how young SWE is, but I think the fact that cybernetics died and computer science was treated more like mathematics than engineering, so software people rarely if ever practice system thinking. It’s also striking that a software engineering undergraduate will never take a course on ethics, or social theory, as part of a core engineering curriculum. Whereas within the mechanical and specifically civil engineering, you will absolutely take courses that show you your social impact on your engineering. That is just never happening in software pedagogy