8 ms·

It seems every time I read an article where some Agile consultant extols the evils of waterfall the author commits a Just In Case Fallacy or the lede is buried.

It seems every time I read an article where some Agile consultant extols the evils of waterfall the author commits a Just In Case Fallacy or the lede is buried. This article is no different: Teams only checked in – wait for it – every three months in a formal schedule review. I listened as Henrich mentioned that the teams complained about the volume of paperwork he makes them fill out for these quarterly reviews. And he was unhappy with the quality of the reports because he felt most teams wrote the reports the night before the review. That's not a problem with AgileFall or ScrumFall, that's ineffective management irrespective of how the sausage is made. But nobody has to wear the Scarlet Letter of Project Failure if we assign blame to the process! I've also never found that pure (like the wind-driven snow capital-A) Agile matches reality. After spending more than a decade wrestling with it - I still haven't figured out how quality assurance and test fits into the same iteration in Agile. Most teams I inherit have non-technical QA resources. Most projects I inherit have complex business logic and integration needs that aren't easily testable until some state of development "done" has been achieved. And time and time again I find myself staggering their effort slightly behind development so that test and development resources aren't taking turns sitting on their asses waiting for the other side. I've written about this before as was even been publicly admonished by an Agile Manifesto signatory (hallowed be their names) that I was doing it WRONG WRONG WRONG! But I was never able to figure out how to contort reality to satisfy their criticism. Believing in iterations and backlogs and prioritization and shortened feedback loops wasn't enough. Thinking QA - any QA! - occurred AFTER any development was done was altogether impure. And despite asking repeatedly for concrete examples of how its done the "right" way, the only responses I received were vague platitudes and consultant sales blurbs that I guess I wasn't smart enough to understand. In the end, I don't think there's any escaping that work has to be done in a sequence: identify a need, identify a solution, implement the solution, validate the solution. And so every software project has at least a little bit of waterfall mixed in.

Groxx 7y ago
My Grand Unifying Theory of Explaining Agile Proselytizers™ is "if it's not working, you're doing it wrong". True Agile always works. Oh, and it has more-frequent check-ins/iterations than Waterfall (which is the defining aspect of Waterfall: fewer check-ins/iterations than Agile). The actual frequency is undefined, it exists only in comparison to Waterfall. --- The problem is that that kind of stance does nothing but fluff up agile consultant's resumes. It doesn't actually lead to improvements that last, or even discussion that might lead to improvements. Agile is perfect because it's Agile.
- collyw 7y ago
  This makes me smile so much. As I have commented before no one can actually agree what Agile is. someone always says it doesn't work for them. Someone always replies with "your doing it wrong" with absolutely no indication of how to do it right. Thankfully the agile trend seems to be left out of interview questions these days. "Do you do Agile?" - "What do you define as Agile?" -"Does anyone NOT do agile?" -"Do you think it actually makes a difference?" would be the responses that cam to mind. Apparently I didn't get a job based on a variation of the last one. They thought I would be difficult to work with.
  - walshemj 7y ago
    The answer is YES I have, the more interesting Q is what sort of agile does your company do or more to the point "says it does vs reality".
- dragonwriter 7y ago
  > My Grand Unifying Theory of Explaining Agile Proselytizers™ is "if it's not working, you're doing it wrong". If it's not working, but you keep trying to do it the same way (whether because management tells you to or because a book or consultant does), you are doing it wrong. Because Agile is very little except the team continuously adapting process based on observation of what does and doesn't work for them. (Waterfall can be Agile, but only if it is waterfall adopted because the team views it as the best way to deliver working software in their particular circumstance and is continuously open to reevaluating that conclusion based on evidence. Agile isn't opposed to waterfall but to the way Waterfall was imposed. Which is also the way Scrum and variants are now frequently imposed, producing the same kinds of problems of process/problem fit.)
  - Jtsummers 7y ago
    I suspect one of the key things missing in most people's understanding is that agile is part of the "continuous process improvement" camp. It is a set of principles, values, and practices which should be examined, in addition to the work being produced, and improved over time. Sterman's book Business Dynamics has a great diagram that I think is relevant (I saw it there, but it's probably in other books on system dynamics and related fields). I'll attempt to ascii art it here: Mental model -> Strategy/Structure/Decision Rules | v Decisions ^ \ / \ / v Information <----- Real World Feedback That is most people's and organization's view of the world. They have a model, they develop strategies (practices, methods), then they implement them. Feedback from the real world informs their next decision, but they don't go back and reevaluate the initial model and strategy. A learning organization, continuous improvement, agile, whatever you want adds an additional loop. The information feedback gets pushed back into the mental model: Mental model -> Strategy/Structure/Decision Rules ^ | | v | Decisions | ^ \ | / \ | / v Information <------ Real World Feedback The Agile Manifesto, in the twelfth principle, emphasizes this aspect: At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly. This is not unique to agile, it is just emphasized in it. This is not a superfluous detail, this is actually critical, and commonly missed.
alchemism 7y ago
The in-house coach pushed back mightily against hiring an engineer with a QA background with pretty much the exact same reasoning. The testing would be done in the sprint by the team, so there was no reason to proceed with the hire. The reality of the production code clearly showed a different reality, but the attempts at persuasion were made anyway. I observed one of the Manifesto signatories for the first time last night. Superb presentation skills, I'll give them that. I'm more of a Taoist than a Confucian (and a sysadmin to boot) in this sense: The system which can be defined is not the true system.
- Yhippa 7y ago
  I wonder how much we've lost as end-users dealing with completely broken software due to Agile cargo-culting and the increasingly diminished (now gone) role of dedicated QA? > The system which can be defined is not the true system. What do you mean by this?
  - alchemism 7y ago
    Dogmatic structures are by their nature opposed to the qualities of flexibility and adaptability, or in another word, agility.
kerkeslager 7y ago
In defense of agile as I perceive it: As far as I am concerned, everything you need to know about agile is on this page: https://agilemanifesto.org/ https://agilemanifesto.org/ So looking at this: > After spending more than a decade wrestling with it - I still haven't figured out how quality assurance and test fits into the same iteration in Agile. Most teams I inherit have non-technical QA resources. Most projects I inherit have complex business logic and integration needs that aren't easily testable until some state of development "done" has been achieved. And time and time again I find myself staggering their effort slightly behind development so that test and development resources aren't taking turns sitting on their asses waiting for the other side. That sounds like you're following "Individuals and interactions over processes and tools" and "Responding to change over following a plan" quite well. The "QA while you develop" process wasn't working for the skillset of the individuals on your team, and so you prioritized the individuals over the process, and responded to the change of understanding instead of following the plan you now knew was bad. So, public admonishment of Agile Manifesto signatories be damned, you were doing agile, and anyone (including Agile Manifesto signatories) who tells you otherwise can be told to go read the Agile Manifesto as many times as they need to in order to reach enlightenment. The problem is, money ruins everything. Once you sign your name to the Agile Manifesto and the Agile Manifesto becomes famous, the tendency is to try to capitalize on that. And the ways you capitalize on it is by pretending you hold some special knowledge that you can teach in a workshop or give a certificate in it. However, the Agile Manifesto isn't some special knowledge. It's admirable because it articulated an important set of ideals succinctly and clearly, but it's really not anything that a moderately intelligent developer doesn't intuit with enough experience, even if they can't so clearly articulate it. And you can't really teach it in a workshop or give a certificate in it, because the application is different for every team. The way you apply agile is by getting intimately familiar with the individuals involved in the team and slowly adapting the process to fit their needs: you can't choose individuals and interactions over processes and tools if you don't know what the individuals and interactions are. The agile manifesto may be about individuals and interactions over processes and tools, but processes and tools are a lot easier to sell. This means that even if you came across the Agile Manifesto and really grokked it, and maybe even signed your name to it, you've got a powerful incentive to then go forget everything you've learned. In a broader sense, this is why ideas have to be evaluated on their own, irrespective of who those ideas came from. Ideas can be perfect--humans can't.
walshemj 7y ago
I am not sure Steve Blank is just another by the numbers Agile guru with very little development experience of their own.
dragonwriter 7y ago
> I've also never found that pure (like the wind-driven snow capital-A) Agile matches reality. Pure Agile (the Manifesto and Principles) are all about adapting to reality. Canned process sold as “Agile” probably won't work without adaptation to your concrete circumstances, but that problem that invariably comes with using canned process rather than team-owned process adapted to the team and their concrete circumstances is exactly the problem that motivated Agile. That some of the signatories of the Manifesto are now on the dark side peddling canned, one-size-fits-all (but none very well!) process is annoying, sure. > I still haven't figured out how quality assurance and test fits into the same iteration in Agile Sometimes it won't. From your description, the concrete composition of your team with regard to those functions sounds problematic for that. In the long term you might be able to evolve your team to alleviate that, and maybe that is desirable, but in the near term, well, “people and interactions over processes and tools”. Forget what any guru tells you about the right way to do Agile if it doesn't work for your team dealing with the problems you face, and especially if they are just telling you your way is wrong without even proposing a testable alternative. Agile isn't canned process, it's the rejection of canned process in favor of empowering the people actually doing the work.
- RogerL 7y ago
  Very few people have that flexibility of mind, in my experience. It is in the manifesto "while there is value in the items on the right, we value the items on the left more". I (may) be able to point out you failed here. Agile doesn't reject anything, including process. It just states that it values some things more than others. Process is often vital. I used to write avionics software for the military. As you might guess "move fast and break things" was not our mode of operation. Retrofitting a helicopter requires massive amounts of process and waterfall. You can't not follow a plan. You have to decide where you are putting the electronics, and the specs on rack sizes, power, etc. Then you design the electronics to that. Then it needs to be manufactured, and so on. You can't decide at the flight readiness review that you want a screen on your radio that displays the weather. This is really old stuff. I was taught this (agile way of thinking) in college in 1983 by an electronics professor. He stratified it differently, talking about goals, strategies, and tactics. People get hung up on tactics - "all code must use lower snake_case variables", or "stand up can't be a status report", or, .... But those are all tactics, in service of a goal. You need to remain flexible he lectured (and I agree), and recognize when the goals have changed, and/or when tactics and strategies conflict or make things non-optimal. That's all pretty much what the agile manifesto says, except they impose a weighting on values that I don't think fits all cases. As in my example above, sometimes process is more important, despite the many costs that kind of thing incurs. Cause, you know, we don't want people to die when they fly the helicopter. And in that helicopter retrofit, the SW lead (Hi Paul!) had a saying "design a little, code a little, test a little". It was never waterfall, even back then, though you were in a waterfall process necessitated by the reality of complex projects. It was all about responding flexibly where you can, and being rigid when you can't. You don't push a sw patch 2 days before the flight readiness review. You don't decide to add components to a radio if it exceeds the designed power specs of the power network in the airframe. And so on. You may be able to change a screen element on a flight computer if the pilot suggests something is confusing or sub-optimal, but even that has high costs (revise all the training materials and documentation, simulators, etc, run all pilots through the training, it can run into millions). Always think flexibly about why you are making these decisions, why you are doing this, why you, say, empower people vs canned process, as you stated. Without that, you are just back to unthinking process, claiming to be agile while you are anything but. Back to how I started - I have yet to come across anyone running around extolling a specific methodology, including agile, that was anything but unthinking, inflexible, and unable to answer real world questions. It's just particularly ironic given the name 'agile'. Edit: I left something unsaid, and it's important. The Agile manifesto is at the level of 'strategy', not goal. It is not ever our goal to "respond to change", as it states. That's a strategy. Our goal may or mat not make that strategy possible or desirable. I don't want to kill people, so if I'm making a helicopter change has to be planned out very carefully. Agile is mostly a set of strategies aimed to get working software into the hands of clients as soon as possible because things like specifying what you want, pricing out development costs, and so on are so hard. That often just doesn't hold. While I can't disagree with what is written given the states objectives, I think it is just another somewhat inflexible way of thinking about systems. It works for a lot of bay area type development, but goes off the rails in many other domains if you don't recognize the values listed, and the relative weightings, are based on a number of assumptions about what your real goals are.
dwd 7y ago
The truth is - to be agile you embrace Waterfall if that's the best approach for the project/team/client. Agile is about being flexible and adaptable in your approach in order to get the job done. Anyone who tells you it's anything specific is trying to sell you something.

Groxx 7y ago

My Grand Unifying Theory of Explaining Agile Proselytizers™ is "if it's not working, you're doing it wrong". True Agile always works. Oh, and it has more-frequent check-ins/iterations than Waterfall (which is the defining aspect of Waterfall: fewer check-ins/iterations than Agile). The actual frequency is undefined, it exists only in comparison to Waterfall. --- The problem is that that kind of stance does nothing but fluff up agile consultant's resumes. It doesn't actually lead to improvements that last, or even discussion that might lead to improvements. Agile is perfect because it's Agile.

collyw 7y ago
This makes me smile so much. As I have commented before no one can actually agree what Agile is. someone always says it doesn't work for them. Someone always replies with "your doing it wrong" with absolutely no indication of how to do it right. Thankfully the agile trend seems to be left out of interview questions these days. "Do you do Agile?" - "What do you define as Agile?" -"Does anyone NOT do agile?" -"Do you think it actually makes a difference?" would be the responses that cam to mind. Apparently I didn't get a job based on a variation of the last one. They thought I would be difficult to work with.
- walshemj 7y ago
  The answer is YES I have, the more interesting Q is what sort of agile does your company do or more to the point "says it does vs reality".
dragonwriter 7y ago
> My Grand Unifying Theory of Explaining Agile Proselytizers™ is "if it's not working, you're doing it wrong". If it's not working, but you keep trying to do it the same way (whether because management tells you to or because a book or consultant does), you are doing it wrong. Because Agile is very little except the team continuously adapting process based on observation of what does and doesn't work for them. (Waterfall can be Agile, but only if it is waterfall adopted because the team views it as the best way to deliver working software in their particular circumstance and is continuously open to reevaluating that conclusion based on evidence. Agile isn't opposed to waterfall but to the way Waterfall was imposed. Which is also the way Scrum and variants are now frequently imposed, producing the same kinds of problems of process/problem fit.)
- Jtsummers 7y ago
  I suspect one of the key things missing in most people's understanding is that agile is part of the "continuous process improvement" camp. It is a set of principles, values, and practices which should be examined, in addition to the work being produced, and improved over time. Sterman's book Business Dynamics has a great diagram that I think is relevant (I saw it there, but it's probably in other books on system dynamics and related fields). I'll attempt to ascii art it here: Mental model -> Strategy/Structure/Decision Rules | v Decisions ^ \ / \ / v Information <----- Real World Feedback That is most people's and organization's view of the world. They have a model, they develop strategies (practices, methods), then they implement them. Feedback from the real world informs their next decision, but they don't go back and reevaluate the initial model and strategy. A learning organization, continuous improvement, agile, whatever you want adds an additional loop. The information feedback gets pushed back into the mental model: Mental model -> Strategy/Structure/Decision Rules ^ | | v | Decisions | ^ \ | / \ | / v Information <------ Real World Feedback The Agile Manifesto, in the twelfth principle, emphasizes this aspect: At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly. This is not unique to agile, it is just emphasized in it. This is not a superfluous detail, this is actually critical, and commonly missed.

alchemism 7y ago

The in-house coach pushed back mightily against hiring an engineer with a QA background with pretty much the exact same reasoning. The testing would be done in the sprint by the team, so there was no reason to proceed with the hire. The reality of the production code clearly showed a different reality, but the attempts at persuasion were made anyway. I observed one of the Manifesto signatories for the first time last night. Superb presentation skills, I'll give them that. I'm more of a Taoist than a Confucian (and a sysadmin to boot) in this sense: The system which can be defined is not the true system.

Yhippa 7y ago
I wonder how much we've lost as end-users dealing with completely broken software due to Agile cargo-culting and the increasingly diminished (now gone) role of dedicated QA? > The system which can be defined is not the true system. What do you mean by this?
- alchemism 7y ago
  Dogmatic structures are by their nature opposed to the qualities of flexibility and adaptability, or in another word, agility.

kerkeslager 7y ago

In defense of agile as I perceive it: As far as I am concerned, everything you need to know about agile is on this page: https://agilemanifesto.org/ https://agilemanifesto.org/ So looking at this: > After spending more than a decade wrestling with it - I still haven't figured out how quality assurance and test fits into the same iteration in Agile. Most teams I inherit have non-technical QA resources. Most projects I inherit have complex business logic and integration needs that aren't easily testable until some state of development "done" has been achieved. And time and time again I find myself staggering their effort slightly behind development so that test and development resources aren't taking turns sitting on their asses waiting for the other side. That sounds like you're following "Individuals and interactions over processes and tools" and "Responding to change over following a plan" quite well. The "QA while you develop" process wasn't working for the skillset of the individuals on your team, and so you prioritized the individuals over the process, and responded to the change of understanding instead of following the plan you now knew was bad. So, public admonishment of Agile Manifesto signatories be damned, you were doing agile, and anyone (including Agile Manifesto signatories) who tells you otherwise can be told to go read the Agile Manifesto as many times as they need to in order to reach enlightenment. The problem is, money ruins everything. Once you sign your name to the Agile Manifesto and the Agile Manifesto becomes famous, the tendency is to try to capitalize on that. And the ways you capitalize on it is by pretending you hold some special knowledge that you can teach in a workshop or give a certificate in it. However, the Agile Manifesto isn't some special knowledge. It's admirable because it articulated an important set of ideals succinctly and clearly, but it's really not anything that a moderately intelligent developer doesn't intuit with enough experience, even if they can't so clearly articulate it. And you can't really teach it in a workshop or give a certificate in it, because the application is different for every team. The way you apply agile is by getting intimately familiar with the individuals involved in the team and slowly adapting the process to fit their needs: you can't choose individuals and interactions over processes and tools if you don't know what the individuals and interactions are. The agile manifesto may be about individuals and interactions over processes and tools, but processes and tools are a lot easier to sell. This means that even if you came across the Agile Manifesto and really grokked it, and maybe even signed your name to it, you've got a powerful incentive to then go forget everything you've learned. In a broader sense, this is why ideas have to be evaluated on their own, irrespective of who those ideas came from. Ideas can be perfect--humans can't.

walshemj 7y ago

I am not sure Steve Blank is just another by the numbers Agile guru with very little development experience of their own.

dragonwriter 7y ago

> I've also never found that pure (like the wind-driven snow capital-A) Agile matches reality. Pure Agile (the Manifesto and Principles) are all about adapting to reality. Canned process sold as “Agile” probably won't work without adaptation to your concrete circumstances, but that problem that invariably comes with using canned process rather than team-owned process adapted to the team and their concrete circumstances is exactly the problem that motivated Agile. That some of the signatories of the Manifesto are now on the dark side peddling canned, one-size-fits-all (but none very well!) process is annoying, sure. > I still haven't figured out how quality assurance and test fits into the same iteration in Agile Sometimes it won't. From your description, the concrete composition of your team with regard to those functions sounds problematic for that. In the long term you might be able to evolve your team to alleviate that, and maybe that is desirable, but in the near term, well, “people and interactions over processes and tools”. Forget what any guru tells you about the right way to do Agile if it doesn't work for your team dealing with the problems you face, and especially if they are just telling you your way is wrong without even proposing a testable alternative. Agile isn't canned process, it's the rejection of canned process in favor of empowering the people actually doing the work.

RogerL 7y ago
Very few people have that flexibility of mind, in my experience. It is in the manifesto "while there is value in the items on the right, we value the items on the left more". I (may) be able to point out you failed here. Agile doesn't reject anything, including process. It just states that it values some things more than others. Process is often vital. I used to write avionics software for the military. As you might guess "move fast and break things" was not our mode of operation. Retrofitting a helicopter requires massive amounts of process and waterfall. You can't not follow a plan. You have to decide where you are putting the electronics, and the specs on rack sizes, power, etc. Then you design the electronics to that. Then it needs to be manufactured, and so on. You can't decide at the flight readiness review that you want a screen on your radio that displays the weather. This is really old stuff. I was taught this (agile way of thinking) in college in 1983 by an electronics professor. He stratified it differently, talking about goals, strategies, and tactics. People get hung up on tactics - "all code must use lower snake_case variables", or "stand up can't be a status report", or, .... But those are all tactics, in service of a goal. You need to remain flexible he lectured (and I agree), and recognize when the goals have changed, and/or when tactics and strategies conflict or make things non-optimal. That's all pretty much what the agile manifesto says, except they impose a weighting on values that I don't think fits all cases. As in my example above, sometimes process is more important, despite the many costs that kind of thing incurs. Cause, you know, we don't want people to die when they fly the helicopter. And in that helicopter retrofit, the SW lead (Hi Paul!) had a saying "design a little, code a little, test a little". It was never waterfall, even back then, though you were in a waterfall process necessitated by the reality of complex projects. It was all about responding flexibly where you can, and being rigid when you can't. You don't push a sw patch 2 days before the flight readiness review. You don't decide to add components to a radio if it exceeds the designed power specs of the power network in the airframe. And so on. You may be able to change a screen element on a flight computer if the pilot suggests something is confusing or sub-optimal, but even that has high costs (revise all the training materials and documentation, simulators, etc, run all pilots through the training, it can run into millions). Always think flexibly about why you are making these decisions, why you are doing this, why you, say, empower people vs canned process, as you stated. Without that, you are just back to unthinking process, claiming to be agile while you are anything but. Back to how I started - I have yet to come across anyone running around extolling a specific methodology, including agile, that was anything but unthinking, inflexible, and unable to answer real world questions. It's just particularly ironic given the name 'agile'. Edit: I left something unsaid, and it's important. The Agile manifesto is at the level of 'strategy', not goal. It is not ever our goal to "respond to change", as it states. That's a strategy. Our goal may or mat not make that strategy possible or desirable. I don't want to kill people, so if I'm making a helicopter change has to be planned out very carefully. Agile is mostly a set of strategies aimed to get working software into the hands of clients as soon as possible because things like specifying what you want, pricing out development costs, and so on are so hard. That often just doesn't hold. While I can't disagree with what is written given the states objectives, I think it is just another somewhat inflexible way of thinking about systems. It works for a lot of bay area type development, but goes off the rails in many other domains if you don't recognize the values listed, and the relative weightings, are based on a number of assumptions about what your real goals are.

dwd 7y ago

The truth is - to be agile you embrace Waterfall if that's the best approach for the project/team/client. Agile is about being flexible and adaptable in your approach in order to get the job done. Anyone who tells you it's anything specific is trying to sell you something.