6 ms·

I agree with you, except I’d say that the ideal of Agile, the manifesto itself, responding to change, etc., is a significantly worse workflow process than Water

by mlthoughts2018 8y ago

I agree with you, except I’d say that the ideal of Agile, the manifesto itself, responding to change, etc., is a significantly worse workflow process than Waterfall (which is also bad). Agile prescribes so many things that are absolutely pushover, toothless concepts in the face of basic corporate bureaucracy that it really is a case where it’s the tool’s own fault that everyone uses it wrong. I’ve seen Waterfall used successfully. Not often, but on a few defense research projects it was great. I have never once seen or heard of any Agile process anywhere actually working with any degree of success. I’ve heard plenty of people indulging some No True Scotsman nonsense about how a “real” Agile process can succeed, and then proceeding to write a diatribe about their “real Agile” workplace that is clearly so fucked with dysfunction that the person writing can’t even realize they are only shipping software in spite of Agile and not the least bit because of it. But I’ve never ever seen or heard of it actually causing success, not across any of the countless blog posts or anecdata on HN, and not across 6 different jobs at companies of various ages, sizes and capitalization levels all using Agile withvarying degrees of formality, pragmatism, Agile coaching and consulting, etc. None of it made any difference. Agile just hinders people. It is not capable of doing otherwise. Now, having said all that, I do agree that for processes that can work, processes that are not overly prescriptive, let different teams do things different ways at different times based on different goals or different personnel, let teams ignore the customer when needed, enforce long-term planning when needed, do pure research with no time boxing when needed, etc., then for those kinds of common sense, non-Agile processes, we have to change the incentives so that people are aligned to care about the objective truth and objective outcomes required to get the job done.

com2kid 8y ago
> I have never once seen or heard of any Agile process anywhere actually working with any degree of success. I ran a team were we shipped (firmware!) every 3 weeks. After the first year of tracking all our numbers we were able to hit our estimates within a few hours +/- for the entire team. 3 week sprints, 2 weeks writing code, 1 week testing and bug fixing. It wasn't perfect, and learning how to do it was painful, but once we got going it worked great. We didn't adhere strongly to any one guiding principle, but being able to use sprints to align work across multiple teams, and as a planning mechanism for management was incredibly useful. "Our breakdown of that feature means it'll take 3 sprints in total, the last of those sprints the work is split across 2 devs, if one of those devs gets reassigned it'll take 4 sprints." Being forced to actually plan work out, and then being held accountable for the hours, meant that over time we got good at estimating, and knowing how far to break work down. There is of course a trade off between gains from breaking work down, and time spent doing the breakdown! Several months were rather painful when we went too far one way, and then inevitably the next few months swung too far in the other direction. But in the end? Yeah it all worked out. We figured out what length of sprint worked for us, what type of estimation system to use, and how much work to put into our planning sessions. FWIW one of the huge benefit of doing the work breakdowns is being able to see opportunities to parallelize work, and also to see natural stopping points. My team was lucky to have someone almost full-time dedicated to managing our workflow, without that the entire process wouldn't have been tenable. The tools just suck up too much time.
- mlthoughts2018 8y ago
  > “Being forced to actually plan work out, and then being held accountable for the hours, meant that over time we got good at estimating” I’m skeptical of this, because even working in a team that had many person-decades of experience of estimating software delivery, the consensus opinion was that nobody’s estimates are worth anything and that the business would actually be worse off using estimates even as mere guidelines than just working on things, hitting blockers, and adjust expectations as you go. But regardless it sounds like your process was fairly different from Agile. The only superficial similarity was having some notion of a regular sprint cadence, but cadence based scheduling is common to many project management strategies. It sounds like you had something very pragmatic for your exact situation, which is great — that’s the whole goal, the very thing Agile paradoxically does not conceptually support.
  - com2kid 8y ago
    > I’m skeptical of this, because even working in a team that had many person-decades of experience of estimating software delivery, the consensus opinion was that nobody’s estimates are worth anything and that the business would actually be worse off using estimates even as mere guidelines than just working on things, hitting blockers, and adjust expectations as you go. I usually agree, but we were in a bit of a special situation: 1. In house designed and built custom RTOS 2. In house designed and built custom hardware platform 3. Our entire UI framework was highly customized by us, for our specific needs It turns out, when you control all the variables, making estimations becomes possible. It isn't easy, but it is possible! > the very thing Agile paradoxically does not conceptually support. Early on, a few of the more senior engineers wanted everything to be done as one big design up front. That is after all how embedded hardware projects are normally delivered. (And for hardware that makes a lot of sense, lead times are a thing!) The idea of shipping embedded firmware every 3 weeks is considered insane by many people. Heck I think that throughout the industry it is still considered insane. But if we had not pushed for "agile", users would have suffered. They wouldn't have gotten the incredible stream of features that they did, and the teams wouldn't have been forced to maintain constant stability and code quality. That idea of "always ready to ship" is one of the tenants of agile. Of course having milestones is common, but "our code is always good" is not common, and it is one of agile's differentiators and adhering to that leads to systematic changes in how teams function.
hoorayimhelping 8y ago
>I have never once seen or heard of any Agile process anywhere actually working with any degree of success. I worked on two internal tools at Etsy that used the most agile of agile processes (no product manager or project manager, no estimation, no timelines and burndown charts, just the people who used the tools and the people writing the tools, a big physical board with cards on it representing the work that needs to get done in priority ordering, and daily standups with the manager of the engineering team. The whole team would have retros every two weeks that were kind of social, kind of project management, kind of technical. They were the two most wildly successful projects I've ever worked on and the tools were well loved and last I checked a couple years ago, at least one was still being used about 4 years on. They solved the users' problems extremely well.
- mlthoughts2018 8y ago
  It’s hilarious for me to hear this about Etsy. I live not far in Brooklyn and have several friends who have worked there. I specifically did not apply to a machine learning role at Etsy after talking to them about the software engineering culture there. Maybe I received wrong data, but according to their reports anyway, it sounded like Etsy would be almost a case study in bad Agile processes.
  - sooheon 8y ago
    I feel like you could be very right or very wrong, but have no way to tell because you are emphatic without being specific. What are the indicators of bad agile processes, and what would be better processes?
wpietri 8y ago
No process "causes success", so if that's your standard, nothing will ever meet it. Also, there is no single "Agile process" that is actually Agile. The point is for teams to continuously improve over time in a direction that shortens feedback loops, decreases waste, and improves quality. Anyhow, I've used these approaches very happily in a startup context. In particular, I've used processes derived from Extreme Programming, so we used practices like pair programming, frequent pair rotation, test-driven development, small units of work, continuous deployment, weekly retrospectives, continuous process improvement, and a number of other things. A+++, would do again. And all the developers and business people said the same thing. Part of the problem you may be having is that most of what people call "Agile" is what I'd call "mini-waterfall". It's where a waterfall-oriented organization decided Agile was fashionable so they borrowed a bunch of labels, cargo-culted some rituals, turned up their release frequency slightly, and declared themselves awesome. I agree this is shitty nonsense. But we didn't trademark the term "Agile" at the beginning, so anybody can use the term to mean anything they like. I think this slide began circa 2007, and in 2011 I wrote about what a mess things were: http://agilefocus.com/2011/02/21/agiles-second-chasm-and-how-we-fell-in/ http://agilefocus.com/2011/02/21/agiles-second-chasm-and-how...
jdlshore 8y ago
> I have never once seen or heard of any Agile process anywhere actually working with any degree of success. Then either your experience is narrow... > I’ve heard plenty of people [talk about how their] Agile process can succeed... that the person writing can’t even realize they are only shipping software in spite of Agile. Agile just hinders people. It is not capable of doing otherwise. ...or your mind is.
dragonwriter 8y ago
> Agile prescribes so many things that are absolutely pushover, toothless concepts in the face of basic corporate bureaucracy Agile prescribes exactly one thing; adapting processes and tools to use those that work for producing working software for the actual people on the team, working in their actual context. > I’ve seen Waterfall used successfully. And pretty much all the concrete methods sold as “Agile” are processes that have been used successfully somewhere, too, that's usually what motivates documenting them. Moreover, Lean (which is essentially the system in which the values expressed in the Agile Manifesto are concretely operationalized, unlike what happens in the “Agile” community where you have the Manifesto reject rote application of canned processes, and then the literature of the community is almost entirely canned processes for rote application rather than discussion of how to adapt processes to teams and situations) shows how Agile values can work, as well as “Agile” canned processes.
- mlthoughts2018 8y ago
  > “And pretty much all the concrete methods sold as “Agile” are processes that have been used successfully somewhere, too, that's usually what motivates documenting them.” Agile is a gimmick to sell. That’s what motivates documenting different examples of so-called Agile. It’s similar to the age old scam of sending out mailers to 2^n different sequences of buy / sell advice for n different investment decisions, and then only focusing on the people who received the advice that turned out to be correct in every case. Say that some organization “uses Agile” (whatever that means), but that the engineers deliver a product in spite of the processes that exist “because of Agile” and would have delivered better results with less risk had those processes not existed. In that case, Agile will still be labeled as a success story and trumpeted around as a documented example for consultants to sell, despite Agile having a negative effect. It boils down to a causal inference problem: what would have been delivered in the counterfactual situation of some methodology that did not impose any “Agile” processes? I’m just pointing out that in every implementation of Agile I’ve personally worked in, heard about from friends or read about in HN comments (including this thread), all of the details appear to suggest it really is just this counterfactual problem. Agile has a subtractive, hindering effect on a process of delivering software, and due to engineers doing extra work to overcome Agile and deliver things in spite of it, the end result is that, no thanks to Agile, organizations still ship things and make customer’s happy. Just not as happy as if they did not approach work with Agile process management.

com2kid 8y ago

> I have never once seen or heard of any Agile process anywhere actually working with any degree of success. I ran a team were we shipped (firmware!) every 3 weeks. After the first year of tracking all our numbers we were able to hit our estimates within a few hours +/- for the entire team. 3 week sprints, 2 weeks writing code, 1 week testing and bug fixing. It wasn't perfect, and learning how to do it was painful, but once we got going it worked great. We didn't adhere strongly to any one guiding principle, but being able to use sprints to align work across multiple teams, and as a planning mechanism for management was incredibly useful. "Our breakdown of that feature means it'll take 3 sprints in total, the last of those sprints the work is split across 2 devs, if one of those devs gets reassigned it'll take 4 sprints." Being forced to actually plan work out, and then being held accountable for the hours, meant that over time we got good at estimating, and knowing how far to break work down. There is of course a trade off between gains from breaking work down, and time spent doing the breakdown! Several months were rather painful when we went too far one way, and then inevitably the next few months swung too far in the other direction. But in the end? Yeah it all worked out. We figured out what length of sprint worked for us, what type of estimation system to use, and how much work to put into our planning sessions. FWIW one of the huge benefit of doing the work breakdowns is being able to see opportunities to parallelize work, and also to see natural stopping points. My team was lucky to have someone almost full-time dedicated to managing our workflow, without that the entire process wouldn't have been tenable. The tools just suck up too much time.

mlthoughts2018 8y ago
> “Being forced to actually plan work out, and then being held accountable for the hours, meant that over time we got good at estimating” I’m skeptical of this, because even working in a team that had many person-decades of experience of estimating software delivery, the consensus opinion was that nobody’s estimates are worth anything and that the business would actually be worse off using estimates even as mere guidelines than just working on things, hitting blockers, and adjust expectations as you go. But regardless it sounds like your process was fairly different from Agile. The only superficial similarity was having some notion of a regular sprint cadence, but cadence based scheduling is common to many project management strategies. It sounds like you had something very pragmatic for your exact situation, which is great — that’s the whole goal, the very thing Agile paradoxically does not conceptually support.
- com2kid 8y ago
  > I’m skeptical of this, because even working in a team that had many person-decades of experience of estimating software delivery, the consensus opinion was that nobody’s estimates are worth anything and that the business would actually be worse off using estimates even as mere guidelines than just working on things, hitting blockers, and adjust expectations as you go. I usually agree, but we were in a bit of a special situation: 1. In house designed and built custom RTOS 2. In house designed and built custom hardware platform 3. Our entire UI framework was highly customized by us, for our specific needs It turns out, when you control all the variables, making estimations becomes possible. It isn't easy, but it is possible! > the very thing Agile paradoxically does not conceptually support. Early on, a few of the more senior engineers wanted everything to be done as one big design up front. That is after all how embedded hardware projects are normally delivered. (And for hardware that makes a lot of sense, lead times are a thing!) The idea of shipping embedded firmware every 3 weeks is considered insane by many people. Heck I think that throughout the industry it is still considered insane. But if we had not pushed for "agile", users would have suffered. They wouldn't have gotten the incredible stream of features that they did, and the teams wouldn't have been forced to maintain constant stability and code quality. That idea of "always ready to ship" is one of the tenants of agile. Of course having milestones is common, but "our code is always good" is not common, and it is one of agile's differentiators and adhering to that leads to systematic changes in how teams function.

hoorayimhelping 8y ago

>I have never once seen or heard of any Agile process anywhere actually working with any degree of success. I worked on two internal tools at Etsy that used the most agile of agile processes (no product manager or project manager, no estimation, no timelines and burndown charts, just the people who used the tools and the people writing the tools, a big physical board with cards on it representing the work that needs to get done in priority ordering, and daily standups with the manager of the engineering team. The whole team would have retros every two weeks that were kind of social, kind of project management, kind of technical. They were the two most wildly successful projects I've ever worked on and the tools were well loved and last I checked a couple years ago, at least one was still being used about 4 years on. They solved the users' problems extremely well.

mlthoughts2018 8y ago
It’s hilarious for me to hear this about Etsy. I live not far in Brooklyn and have several friends who have worked there. I specifically did not apply to a machine learning role at Etsy after talking to them about the software engineering culture there. Maybe I received wrong data, but according to their reports anyway, it sounded like Etsy would be almost a case study in bad Agile processes.
- sooheon 8y ago
  I feel like you could be very right or very wrong, but have no way to tell because you are emphatic without being specific. What are the indicators of bad agile processes, and what would be better processes?

wpietri 8y ago

No process "causes success", so if that's your standard, nothing will ever meet it. Also, there is no single "Agile process" that is actually Agile. The point is for teams to continuously improve over time in a direction that shortens feedback loops, decreases waste, and improves quality. Anyhow, I've used these approaches very happily in a startup context. In particular, I've used processes derived from Extreme Programming, so we used practices like pair programming, frequent pair rotation, test-driven development, small units of work, continuous deployment, weekly retrospectives, continuous process improvement, and a number of other things. A+++, would do again. And all the developers and business people said the same thing. Part of the problem you may be having is that most of what people call "Agile" is what I'd call "mini-waterfall". It's where a waterfall-oriented organization decided Agile was fashionable so they borrowed a bunch of labels, cargo-culted some rituals, turned up their release frequency slightly, and declared themselves awesome. I agree this is shitty nonsense. But we didn't trademark the term "Agile" at the beginning, so anybody can use the term to mean anything they like. I think this slide began circa 2007, and in 2011 I wrote about what a mess things were: http://agilefocus.com/2011/02/21/agiles-second-chasm-and-how-we-fell-in/ http://agilefocus.com/2011/02/21/agiles-second-chasm-and-how...

jdlshore 8y ago

> I have never once seen or heard of any Agile process anywhere actually working with any degree of success. Then either your experience is narrow... > I’ve heard plenty of people [talk about how their] Agile process can succeed... that the person writing can’t even realize they are only shipping software in spite of Agile. Agile just hinders people. It is not capable of doing otherwise. ...or your mind is.

dragonwriter 8y ago

> Agile prescribes so many things that are absolutely pushover, toothless concepts in the face of basic corporate bureaucracy Agile prescribes exactly one thing; adapting processes and tools to use those that work for producing working software for the actual people on the team, working in their actual context. > I’ve seen Waterfall used successfully. And pretty much all the concrete methods sold as “Agile” are processes that have been used successfully somewhere, too, that's usually what motivates documenting them. Moreover, Lean (which is essentially the system in which the values expressed in the Agile Manifesto are concretely operationalized, unlike what happens in the “Agile” community where you have the Manifesto reject rote application of canned processes, and then the literature of the community is almost entirely canned processes for rote application rather than discussion of how to adapt processes to teams and situations) shows how Agile values can work, as well as “Agile” canned processes.

mlthoughts2018 8y ago
> “And pretty much all the concrete methods sold as “Agile” are processes that have been used successfully somewhere, too, that's usually what motivates documenting them.” Agile is a gimmick to sell. That’s what motivates documenting different examples of so-called Agile. It’s similar to the age old scam of sending out mailers to 2^n different sequences of buy / sell advice for n different investment decisions, and then only focusing on the people who received the advice that turned out to be correct in every case. Say that some organization “uses Agile” (whatever that means), but that the engineers deliver a product in spite of the processes that exist “because of Agile” and would have delivered better results with less risk had those processes not existed. In that case, Agile will still be labeled as a success story and trumpeted around as a documented example for consultants to sell, despite Agile having a negative effect. It boils down to a causal inference problem: what would have been delivered in the counterfactual situation of some methodology that did not impose any “Agile” processes? I’m just pointing out that in every implementation of Agile I’ve personally worked in, heard about from friends or read about in HN comments (including this thread), all of the details appear to suggest it really is just this counterfactual problem. Agile has a subtractive, hindering effect on a process of delivering software, and due to engineers doing extra work to overcome Agile and deliver things in spite of it, the end result is that, no thanks to Agile, organizations still ship things and make customer’s happy. Just not as happy as if they did not approach work with Agile process management.