9 ms·

An epic treatise on scheduling, bug tracking, and triage

183 points by lostdog 9y ago

PaulRobinson 9y ago
TL;DR: Agile is silly but has some lessons to teach. Most people do it wrong though. Make decisions early and stick to them. Talk about effort not time. Make stories about users. Treat bugs differently to stories (they are on average all the same size, are written from a different perspective, etc.). Abandon stand-ups, steal things from SCRUM and Kanban, and don't talk about deadlines, ever.
- ndh2 9y ago
  I don't think a tl;dr like this is appropriate for this one, because it leaves out the best bits. I mean, this is a very long writeup, and you're condensing it to 7 sentences. For example, the interesting part is not that deadlines don't work, but why they don't work. And I feel like the "why" is missing from a lot of advice on methodology: Everybody talks about how, but why are we supposed to do these things in the first place? And maybe we can learn something while trying to find answers to these questions. I've only read half of it, but I liked it a lot. What about you? Did you like it? Why?
  - falsedan 9y ago
    I think a tl;dr is suppose to leave bits out…
    - saltedmd5 9y ago
      Yes, but it is also supposed to communicate the point. Which is not always possible in a tl;dr because if it was the author probably would have just written a single paragraph in the first place and saved a whole lot of time for everyone.
      - smichel17 9y ago
        I have not read the article (yet; it's bookmarked for later), so I'm speaking generally, here. > it was [possible] the author probably would have just written a single paragraph in the first place and saved a whole lot of time for everyone. This does not fit with my experience, even disregarding things like YouTube videos which drag on in order to justify longer ads. Writing concisely is hard, and often much more time consuming than a lengthy brain dump. It's also tempting to elaborate on every point, however tangential. I have to constantly fight this in my own writing; my desire is to be complete, but really I'm drifting off-topic, diluting my point with irrelevance, making it harder to follow. Based on the comments above, it seems like the tl;dr missed important nuance, but that isn't always so.
      - falsedan 9y ago
        I don't understand. A summary should summarise. An explanation should explain, an argument should convince, a historical overview should retell events. Obviously, a summary can't be an explanation + an argument + history. It has to be the most salient points presented alone. Think about an abstract for a paper: it should tell you enough that you can understand if you should read the paper now, and enough context so that you can remember to read it later if your situation changes. If you feel like the tl;dr doesn't pay full respects to the article, that's fine but I feel like not many readers will mind.
        apenwarr 9y ago
        I quite liked the tl;dr, except that it left out how most of the article is "...and why". We've all heard the advice, but the article is trying to explain why the advice is true. I think an ideal summary would mention that.
        falsedan 9y ago
        OK, I understand. Though I think the article did a terrible job of explaining why: like, why do deadlines cause engineers to become affected by Student Syndrome? It just mentions it at a superficial level, which is perfectly fine for its purposes. edit: also thanks for writing that article, it's great.
- pjc50 9y ago
  So .. what is the business substitute for deadlines? Unless you're genuinely a startup with unlimited funding and total freedom, eventually people are going to ask the question "so what are you going to deliver and when?"
  - apenwarr 9y ago
    That's basically what the entire article is about. The (very) short version is it really is possible to predict when things will be done, but if you tell the engineers or negotiate with the engineers (ie. turn it into a deadline), that counterintuitively tends to make the predictions worse instead of better.
  - agentultra 9y ago
    That is often the first question I get asked when trying to explain these concepts to business owners. My answer is that you can plan for work to be done on a deadline but you can't negotiate when it will be done with the people doing the work. It's going to take as long as it takes. It's poor management that sets unrealistic expectations and demands results. You can plan for work by using data. You get data by tracking effort estimated versus completed and triaging bugs. You prioritize goals instead of setting deadlines. You measure and refine. It sounds counter-intuitive to business people who think in terms of, I just sold customer X the product and they need it delivered by Y so that we can get the team paid by Z. This is where poor management decisions can sink your team. If Y is decided by the sales or management team with the customer and they didn't consult their engineering team... then they're working on another planet. The goal of processes like this are not to eliminate Y but to set reasonable expectations and objectives. As I like to remind my business owners: you can have something that works -- it may not be the whole kit -- or you can have nothing at all. Winning is about prioritizing objectives. One book I've read recently that taught me a lot about management is Extreme Ownership[0]. I think there's a lot of cross-over from this book into Agile methodologies that I think non-technical stakeholders can really understand. [0] https://www.goodreads.com/book/show/23848190-extreme-ownership https://www.goodreads.com/book/show/23848190-extreme-ownersh...
    - jasode 9y ago
- peterwwillis 9y ago
  Most people get way too wrapped up in jargon and staying inside the Agile box. The method is just an idea, a loose framework, a suggestion. Executives and management will always try to rigidly implement the method (because Peter Principle) and this should be circumvented whenever possible.
- nine_k 9y ago
  > Talk about effort not time I can't understand this. You can measure time, but you can't measure effort directly, unless you're maybe doing some physical work like pulling weights. Also, you can't usually increase a mental effort, except by spending more time (e.g. by working overtime). I can only see points or other non-time measures as proxies to time intervals (with uncertainty), but we can't know the function points -> time beforehand, we can only measure and map it from experience. If you could explain "effort" from some other angle, I'd be grateful.
  - apenwarr 9y ago
    Yeah, this is why I don't prefer the common "effort vs time" distinction. Instead I like to think of "relative size of task," which is a very easy concept for most people to intuitively understand. It's fairly easy to guess that a task that is twice as "big" will take twice the "effort" and thus also twice the "time," all else being equal. And it turns out that's all we need in order to estimate the schedule. See the detailed commentary here: http://apenwarr.ca/log/?m=201712#slide19 http://apenwarr.ca/log/?m=201712#slide19
apenwarr 9y ago
I’m the author of the article. I apologize for its massive length but wasn’t able to make it shorter without losing my favourite bits :) I’m happy to answer any questions people have here.
- po 9y ago
  Super interesting points here. You mentioned that the real-world Kanban board forces you to empty slots to make room... I've never seen that in any of the software Kanban-style systems have you? I've played around (in spreadsheets and basic apps) with trying to create systems that scaled available slots to team size as a way to force correct granularity.
  - apenwarr 9y ago
    No, I've never seen it enforced by tools. Physical (index cards) kanban boards have an implicit space limit though, and this is one of the too-seldom-acknowledged reasons why they work as well as they do. Unfortunately the software clones of physical kanban boards copied the unnecessary part (visual appearance of index cards) and not the necessary part (limited space at each phase).
    - WorldMaker 9y ago
      Rally lets you set card limits in each column, though in practice that always seemed too easy to change (just add a couple more to the limit; just "temporarily" turn off the limit) to be entirely beneficial.
  - yalooze 9y ago
    Are you talking about work-in-progress (WIP) limits? I've seen tools enforce this so you can't pull another story into the current list. Other tools let you pull but highlight the fact that you're going over the WIP limits by making the whole column eg. red. Or do you mean something else?
  - benji-york 9y ago
    I've used LeanKit's Kanban board to enforce per-lane WIP limits. It works quite well and is, in my opinion, one of the most important aspects of using Kanban boards for software development.
vodkaPong 9y ago
I really enjoyed reading this, despite its length! It makes some great points about scheduling that resonated with me, backed up by interesting thought experiments.
peterwwillis 9y ago
> What doesn't work is deciding when you're going to get there. That would be an ETA, which is quite useful in many cases. > Or telling salespeople they need to sell 10% more next quarter. That's a minimum quota. Also useful. > Or telling school teachers they need to improve their standardized test scores. Which is not so much a 'goal' as it is a response to the need for kids to actually learn information. It's a goal in the sense that "you should be doing your job to a minimum degree of proficiency" is a goal. > Or telling engineers they need to launch their project at Whatever Conference (and not a day sooner or a day later). Which is absolutely an arbitrary goal that doesn't have anything to do with the product, but is also good business sense. Sometimes you need stupid goals.
- apenwarr 9y ago
  What all these wrong-goal examples have in common is that people have tried them over and over, and they never work the way you want. The reason they never work is that they don't specify the method. As one of my favourite Deming quotes says, "If you can accomplish a [numerical quality/goal] goal without a [new] method, then why were you not doing it last year? There is only one possible answer: you were goofing off." What ends up happening is that people will cheat the system so they can hit the target you enforced, while sacrificing some other critical thing you forgot to enforce. In the educational system, for example, what happens is "teach to the test," which causes improved standardized test scores at the expense of actually learning things. And, of course, you get schools that outright cheat when scoring the tests: https://www.washingtonpost.com/news/answer-sheet/wp/2015/04/01/how-and-why-convicted-atlanta-teachers-cheated-on-standardized-tests/ https://www.washingtonpost.com/news/answer-sheet/wp/2015/04/... Re: ETAs, those are predictions, not goals. Predictions are good. Most of the article is about the difference between the two.
  - peterwwillis 9y ago
    The goal is not the problem. The method is not the problem. The problem is the problem. You will have to increase the educational system's budget. You will have to raise taxes. You will have to build more classrooms. You will have to hire more teachers. You will even have to feed the damned kids, and improve their home life. There is no "method" that avoids having to do these things. These are the problems you have to solve to reach the intended goal. How you go about solving these problems does not matter. Use whatever method you want. Sweet talk them. Bribe them. Hire a guy named Vito with a baseball bat. The method doesn't matter. Just solve the problems.
watwut 9y ago
"They're sure they can get done in that amount of time, so they take it easy for the first half. Then they get progressively more panicked as the deadline approaches, and they're late anyway, and/or you ship on time ..." I found that the more experienced/older I am, the less this happens. We even finished multiple such tasks sooner then was required - of course contributing factor was that deadlines were sane. I somewhat used to have this tendency when I was younger. Experience makes you better at managing time risks. It is actually one issue I have with cookie cutter agile. Everyone is so micromanaged by the system, that they don't get to get experience needed to learn the above. There are also contributing cultural factors. In many companies, people who stay late last week tend to be praised and rewarded over those who work in predictable speed and kept eye on the deadline from day one (not in my current team). Incentives matter and finishing tasks at the last moment at cost of evenings/weekends makes you a hero. I personally know people, including leads of agile teams, who believe that last moment desperate effort is somehow necessary. Like you are lazy if you dont. That means there will be last moment effort, whether it was needed or not. So if the team is going to be on time, an additional work is found to be done - or more often possible logical steps to make the deadline are not taken (a feature the customer explicitly said is not necessary is done anyway etc). I wish this would be a joke or exaggeration, but I have seen it multiple times already. While details were each time different, it really did boiled down to techies essentially organizing that desperate effort for themselves. It does not matter what the process is, how exactly you estimate etc. As long as not doing the responsible thing is rewarded, people will do that.
markphip 9y ago
Interesting article, I have a few comments/questions Story points - there is a lot in theory I like about these and you hit on those points. But in practice where my teams have struggled is still in the definition of a point. People understand the relative sizing concept but they still want a definition of what one point means and that invariably winds up being some kind of a time unit, which means all points get thought of in time units. What techniques have you used to solve this problem? Defects - some good points were made but you never really discussed how these are managed as work. Stories and defects are getting worked on at the same time, but how exactly? Are defects just lumped in with stories and the PM prioritizes them? I do not think so, but you are not clear in this area. If a PM prioritizes a story but the team spends all of its time resolving defects then how is value being delivered as expected? You seem to have left this out completely, or I missed it. Finally, also on defects, while your points on not estimating them makes some sense, someone has to decide what to work on, and generally you need some idea how long things will take to make that decision. Even in your wording, you acknowledge some defects take a long time to fix, where as others are super-quick and in the end they all average out. Still, someone in the organization still cares about dates and delivery. It made a lot of sense to me where you indicated that the story point estimate can be valuable to the PM in terms of prioritizing. When they see a story with a high estimate that might cause them to lower its priority compared to other stories they can get delivered quicker. But seemingly defect fixing would have to factor into this somewhere too, and then wouldn't the same concept apply? If defects are not estimated, how can the time it will take to resolve the defect inform the decision making process?
- structural 9y ago
  The whole "point" of story points is that they are meaningless until after something has been actually accomplished. After working for a few weeks, it's pretty easy to say something along the lines of "the team got 52 points finished and deployed last week" - this is useful to know, and over time you'll develop an intuition that may be useful for forecasting what's likely to get done over the next few weeks or months on this particular project with this particular team. That's it, that's the most you can do. Most of the other stuff people try to do is so noisy that it makes for nice spreadsheets of the "plan" but bears very little resemblance to reality.

183 points by lostdog 9y ago

TL;DR: Agile is silly but has some lessons to teach. Most people do it wrong though. Make decisions early and stick to them. Talk about effort not time. Make stories about users. Treat bugs differently to stories (they are on average all the same size, are written from a different perspective, etc.). Abandon stand-ups, steal things from SCRUM and Kanban, and don't talk about deadlines, ever.

ndh2 9y ago
I don't think a tl;dr like this is appropriate for this one, because it leaves out the best bits. I mean, this is a very long writeup, and you're condensing it to 7 sentences. For example, the interesting part is not that deadlines don't work, but why they don't work. And I feel like the "why" is missing from a lot of advice on methodology: Everybody talks about how, but why are we supposed to do these things in the first place? And maybe we can learn something while trying to find answers to these questions. I've only read half of it, but I liked it a lot. What about you? Did you like it? Why?
- falsedan 9y ago
  I think a tl;dr is suppose to leave bits out…
  - saltedmd5 9y ago
    Yes, but it is also supposed to communicate the point. Which is not always possible in a tl;dr because if it was the author probably would have just written a single paragraph in the first place and saved a whole lot of time for everyone.
    - smichel17 9y ago
      I have not read the article (yet; it's bookmarked for later), so I'm speaking generally, here. > it was [possible] the author probably would have just written a single paragraph in the first place and saved a whole lot of time for everyone. This does not fit with my experience, even disregarding things like YouTube videos which drag on in order to justify longer ads. Writing concisely is hard, and often much more time consuming than a lengthy brain dump. It's also tempting to elaborate on every point, however tangential. I have to constantly fight this in my own writing; my desire is to be complete, but really I'm drifting off-topic, diluting my point with irrelevance, making it harder to follow. Based on the comments above, it seems like the tl;dr missed important nuance, but that isn't always so.
    - falsedan 9y ago
      I don't understand. A summary should summarise. An explanation should explain, an argument should convince, a historical overview should retell events. Obviously, a summary can't be an explanation + an argument + history. It has to be the most salient points presented alone. Think about an abstract for a paper: it should tell you enough that you can understand if you should read the paper now, and enough context so that you can remember to read it later if your situation changes. If you feel like the tl;dr doesn't pay full respects to the article, that's fine but I feel like not many readers will mind.
      - apenwarr 9y ago
        I quite liked the tl;dr, except that it left out how most of the article is "...and why". We've all heard the advice, but the article is trying to explain why the advice is true. I think an ideal summary would mention that.
        falsedan 9y ago
        OK, I understand. Though I think the article did a terrible job of explaining why: like, why do deadlines cause engineers to become affected by Student Syndrome? It just mentions it at a superficial level, which is perfectly fine for its purposes. edit: also thanks for writing that article, it's great.
pjc50 9y ago
So .. what is the business substitute for deadlines? Unless you're genuinely a startup with unlimited funding and total freedom, eventually people are going to ask the question "so what are you going to deliver and when?"
- apenwarr 9y ago
  That's basically what the entire article is about. The (very) short version is it really is possible to predict when things will be done, but if you tell the engineers or negotiate with the engineers (ie. turn it into a deadline), that counterintuitively tends to make the predictions worse instead of better.
- agentultra 9y ago
  That is often the first question I get asked when trying to explain these concepts to business owners. My answer is that you can plan for work to be done on a deadline but you can't negotiate when it will be done with the people doing the work. It's going to take as long as it takes. It's poor management that sets unrealistic expectations and demands results. You can plan for work by using data. You get data by tracking effort estimated versus completed and triaging bugs. You prioritize goals instead of setting deadlines. You measure and refine. It sounds counter-intuitive to business people who think in terms of, I just sold customer X the product and they need it delivered by Y so that we can get the team paid by Z. This is where poor management decisions can sink your team. If Y is decided by the sales or management team with the customer and they didn't consult their engineering team... then they're working on another planet. The goal of processes like this are not to eliminate Y but to set reasonable expectations and objectives. As I like to remind my business owners: you can have something that works -- it may not be the whole kit -- or you can have nothing at all. Winning is about prioritizing objectives. One book I've read recently that taught me a lot about management is Extreme Ownership[0]. I think there's a lot of cross-over from this book into Agile methodologies that I think non-technical stakeholders can really understand. [0] https://www.goodreads.com/book/show/23848190-extreme-ownership https://www.goodreads.com/book/show/23848190-extreme-ownersh...
  - jasode 9y ago
peterwwillis 9y ago
Most people get way too wrapped up in jargon and staying inside the Agile box. The method is just an idea, a loose framework, a suggestion. Executives and management will always try to rigidly implement the method (because Peter Principle) and this should be circumvented whenever possible.
nine_k 9y ago
> Talk about effort not time I can't understand this. You can measure time, but you can't measure effort directly, unless you're maybe doing some physical work like pulling weights. Also, you can't usually increase a mental effort, except by spending more time (e.g. by working overtime). I can only see points or other non-time measures as proxies to time intervals (with uncertainty), but we can't know the function points -> time beforehand, we can only measure and map it from experience. If you could explain "effort" from some other angle, I'd be grateful.
- apenwarr 9y ago
  Yeah, this is why I don't prefer the common "effort vs time" distinction. Instead I like to think of "relative size of task," which is a very easy concept for most people to intuitively understand. It's fairly easy to guess that a task that is twice as "big" will take twice the "effort" and thus also twice the "time," all else being equal. And it turns out that's all we need in order to estimate the schedule. See the detailed commentary here: http://apenwarr.ca/log/?m=201712#slide19 http://apenwarr.ca/log/?m=201712#slide19

apenwarr 9y ago

I’m the author of the article. I apologize for its massive length but wasn’t able to make it shorter without losing my favourite bits :) I’m happy to answer any questions people have here.

po 9y ago
Super interesting points here. You mentioned that the real-world Kanban board forces you to empty slots to make room... I've never seen that in any of the software Kanban-style systems have you? I've played around (in spreadsheets and basic apps) with trying to create systems that scaled available slots to team size as a way to force correct granularity.
- apenwarr 9y ago
  No, I've never seen it enforced by tools. Physical (index cards) kanban boards have an implicit space limit though, and this is one of the too-seldom-acknowledged reasons why they work as well as they do. Unfortunately the software clones of physical kanban boards copied the unnecessary part (visual appearance of index cards) and not the necessary part (limited space at each phase).
  - WorldMaker 9y ago
    Rally lets you set card limits in each column, though in practice that always seemed too easy to change (just add a couple more to the limit; just "temporarily" turn off the limit) to be entirely beneficial.
- yalooze 9y ago
  Are you talking about work-in-progress (WIP) limits? I've seen tools enforce this so you can't pull another story into the current list. Other tools let you pull but highlight the fact that you're going over the WIP limits by making the whole column eg. red. Or do you mean something else?
- benji-york 9y ago
  I've used LeanKit's Kanban board to enforce per-lane WIP limits. It works quite well and is, in my opinion, one of the most important aspects of using Kanban boards for software development.

vodkaPong 9y ago

I really enjoyed reading this, despite its length! It makes some great points about scheduling that resonated with me, backed up by interesting thought experiments.

peterwwillis 9y ago

> What doesn't work is deciding when you're going to get there. That would be an ETA, which is quite useful in many cases. > Or telling salespeople they need to sell 10% more next quarter. That's a minimum quota. Also useful. > Or telling school teachers they need to improve their standardized test scores. Which is not so much a 'goal' as it is a response to the need for kids to actually learn information. It's a goal in the sense that "you should be doing your job to a minimum degree of proficiency" is a goal. > Or telling engineers they need to launch their project at Whatever Conference (and not a day sooner or a day later). Which is absolutely an arbitrary goal that doesn't have anything to do with the product, but is also good business sense. Sometimes you need stupid goals.

apenwarr 9y ago
What all these wrong-goal examples have in common is that people have tried them over and over, and they never work the way you want. The reason they never work is that they don't specify the method. As one of my favourite Deming quotes says, "If you can accomplish a [numerical quality/goal] goal without a [new] method, then why were you not doing it last year? There is only one possible answer: you were goofing off." What ends up happening is that people will cheat the system so they can hit the target you enforced, while sacrificing some other critical thing you forgot to enforce. In the educational system, for example, what happens is "teach to the test," which causes improved standardized test scores at the expense of actually learning things. And, of course, you get schools that outright cheat when scoring the tests: https://www.washingtonpost.com/news/answer-sheet/wp/2015/04/01/how-and-why-convicted-atlanta-teachers-cheated-on-standardized-tests/ https://www.washingtonpost.com/news/answer-sheet/wp/2015/04/... Re: ETAs, those are predictions, not goals. Predictions are good. Most of the article is about the difference between the two.
- peterwwillis 9y ago
  The goal is not the problem. The method is not the problem. The problem is the problem. You will have to increase the educational system's budget. You will have to raise taxes. You will have to build more classrooms. You will have to hire more teachers. You will even have to feed the damned kids, and improve their home life. There is no "method" that avoids having to do these things. These are the problems you have to solve to reach the intended goal. How you go about solving these problems does not matter. Use whatever method you want. Sweet talk them. Bribe them. Hire a guy named Vito with a baseball bat. The method doesn't matter. Just solve the problems.

watwut 9y ago

"They're sure they can get done in that amount of time, so they take it easy for the first half. Then they get progressively more panicked as the deadline approaches, and they're late anyway, and/or you ship on time ..." I found that the more experienced/older I am, the less this happens. We even finished multiple such tasks sooner then was required - of course contributing factor was that deadlines were sane. I somewhat used to have this tendency when I was younger. Experience makes you better at managing time risks. It is actually one issue I have with cookie cutter agile. Everyone is so micromanaged by the system, that they don't get to get experience needed to learn the above. There are also contributing cultural factors. In many companies, people who stay late last week tend to be praised and rewarded over those who work in predictable speed and kept eye on the deadline from day one (not in my current team). Incentives matter and finishing tasks at the last moment at cost of evenings/weekends makes you a hero. I personally know people, including leads of agile teams, who believe that last moment desperate effort is somehow necessary. Like you are lazy if you dont. That means there will be last moment effort, whether it was needed or not. So if the team is going to be on time, an additional work is found to be done - or more often possible logical steps to make the deadline are not taken (a feature the customer explicitly said is not necessary is done anyway etc). I wish this would be a joke or exaggeration, but I have seen it multiple times already. While details were each time different, it really did boiled down to techies essentially organizing that desperate effort for themselves. It does not matter what the process is, how exactly you estimate etc. As long as not doing the responsible thing is rewarded, people will do that.

markphip 9y ago

Interesting article, I have a few comments/questions Story points - there is a lot in theory I like about these and you hit on those points. But in practice where my teams have struggled is still in the definition of a point. People understand the relative sizing concept but they still want a definition of what one point means and that invariably winds up being some kind of a time unit, which means all points get thought of in time units. What techniques have you used to solve this problem? Defects - some good points were made but you never really discussed how these are managed as work. Stories and defects are getting worked on at the same time, but how exactly? Are defects just lumped in with stories and the PM prioritizes them? I do not think so, but you are not clear in this area. If a PM prioritizes a story but the team spends all of its time resolving defects then how is value being delivered as expected? You seem to have left this out completely, or I missed it. Finally, also on defects, while your points on not estimating them makes some sense, someone has to decide what to work on, and generally you need some idea how long things will take to make that decision. Even in your wording, you acknowledge some defects take a long time to fix, where as others are super-quick and in the end they all average out. Still, someone in the organization still cares about dates and delivery. It made a lot of sense to me where you indicated that the story point estimate can be valuable to the PM in terms of prioritizing. When they see a story with a high estimate that might cause them to lower its priority compared to other stories they can get delivered quicker. But seemingly defect fixing would have to factor into this somewhere too, and then wouldn't the same concept apply? If defects are not estimated, how can the time it will take to resolve the defect inform the decision making process?

structural 9y ago
The whole "point" of story points is that they are meaningless until after something has been actually accomplished. After working for a few weeks, it's pretty easy to say something along the lines of "the team got 52 points finished and deployed last week" - this is useful to know, and over time you'll develop an intuition that may be useful for forecasting what's likely to get done over the next few weeks or months on this particular project with this particular team. That's it, that's the most you can do. Most of the other stuff people try to do is so noisy that it makes for nice spreadsheets of the "plan" but bears very little resemblance to reality.