4 ms·

As a software engineering manager, I put a story-point-based method of task size estimation and team velocity measurement in place about two years ago. A stipul

As a software engineering manager, I put a story-point-based method of task size estimation and team velocity measurement in place about two years ago. A stipulation of this was that at no time would any one developer have their story point velocity assessed. The purpose of this system is to provide the team with a powerful tool that it needs in order to set a realistic timeline on team deliverables, such as a launch milestone or a new feature. The ability of the story point methodology to have predictive power is destroyed when people are incentivized to gamify it.

sidlls 6y ago
I’d argue the predictive power of story point methodology has not been established. There are simply too many variables to make estimates more accurate than “best guess”: 1/ the team’s composition at a point in time (and all the variability that comes with this) 2/ the team’s dependencies (a kind of exponential effect is at work here: the team’s dependencies also have all these issues in variability and so do their dependencies) 3/ the type of work being estimated in the context of the product (even “basic” things like “CRUD” can be quite intricate in some cases) Basically unless the estimation is literally for a known change (e.g. modify source code files X, Y and Z with code changes A, B, and C) its predictive power diminishes substantially. It’s not a powerful tool, it’s a micromanagement technique disguised as a tool to empower ICs.
- steelframe 6y ago
  I completely agree with the impact of the variables you listed. However it's often the case that you need a starting point to estimate timelines. It's never acceptable to tell the decision makers, "We really have no idea how long this will take." Nor is it acceptable to say, "Sure, we'll have it ready in <licked finger in the wind> 8 weeks! Everyone get to work and make that happen (or else)!" Having a concrete plan ("micromanagement?") for getting to the finish line is important, and the story point methodology is an intuitive tool for figuring out what lengths the tasks need to start out with on the timeline.
  - sidlls 6y ago
    Having a concrete plan with estimates is important. Assigning some "point value" to them, which variously means "complexity", "man-day-equivalent", "man-sprint-equivalent", and possibly others depending on (arbitrarily applied) context and individuals involved, is the gateway to micromanagement. It's entirely possible to provide meaningful estimates of both complexity and time-to-implement without resorting to story point formulations or any particular PM framework. "This will take about 6 weeks and has a few dependencies and unknowns that make it moderately risky to take longer." Rephrasing that as "This will take 3 sprints and has a story point value of 8 points per print" doesn't add any value, except from management's perspective as a way to put some "objective" quantification to the problem of estimates and deliverables.
jniedrauer 6y ago
Have story points been useful to you, even when incentives are aligned? I find that they're often orders of magnitude off, in either direction.
- steelframe 6y ago
  My team has been tracking its story-point-based velocity for about 20 months. The team's composition, product, and structure has been largely unchanged. We started tracking when we were newly formed, and we continued to use the same methodology until we shipped GA. When not focused on just one objective and not under deadline pressure, the team's average story point velocity per week was 30. This was when we tracked anything and everything we did. When we've track against a launch, where we only focused on the minimum set of tasks we needed to meet a clearly-defined objective with inter-team dependencies and a deadline, the team's average velocity has been 12. We don't use story points alone in forecasting timelines. What we do is presume a per-dev weekly velocity of (12 / team size). With all of the tasks on a chart, we work with the team to assign tasks to individuals, and we create a strawman schedule for how all the work gets done to meet the timeline. The "length" of each bug is a function of (12 / time size). We track and reassess how all the tasks are coming along on a weekly basis, with the understanding that the "strawman" schedule will end up looking different than we initially project. For example, take a team size of 6. Then the story points per week per dev is (12 / 6 = 2). Story points per weekday is (2 / 5). So for a task of size 3, we expect one dev to take ((5 / 2) * 3 = 7.5) days to complete, and so the length of the task on the timeline is (rounded up) 8 days. Then we bring the "fudge factors" into play. For the devs who are more productive we tend to schedule them back-to-back with the tasks that are more urgent and/or are blockers. For the devs that tend to need more time, we keep a lot of whitespace between the end of their assigned tasks and the deadline. The only time this methodology has gotten us into trouble is when we've caved to pressure by upper management to say we can actually do 15 story points per week or whatever. The most important thing is to measure actual velocity under as similar conditions as you can, and then refuse to accept any projection with a velocity other than you've actually observed. Story points are opaque outside of your team, and upper management is usually satisfied when you translate the story points into the schedule using the method I described.
Fire-Dragon-DoL 6y ago
What about replacing story points with time ranges? That gives you both time (min, max, which is the information you need) as well as the risk (distance between min and max). Then countermeasures could be taken to reduce the risk, until min and max are reasonably close. Sometimes max is +infinity when there is 0 knowledge, but even that information is useful: missing a role or need research.
- steelframe 6y ago
  My team uses story points as an input into a time range function.
  - Fire-Dragon-DoL 6y ago
    How does it split the time from the risk? My main problem with points is the impossibility to represent: - A long task of 5 days that's very well known, it's 5 days of work - A short task that's very uncertain. It could be a one line change or many days of work With time range, you can represent that and it's very visible. With points, you can represent that as the same points, but that does not reflect reality, one of the two estimates is potentially wrong. This also helps providing input on what research tasks should be performed and what's their value.
    - steelframe 6y ago
      My team uses story points to estimate large projects over long periods of time. Sometimes things are overestimated, and other times things are underestimated. However, when you sum them all together, the errors tend to cancel each other out. I've found that upper management doesn't care whether any given task actually takes 3 days when we estimate it at 3 days. They just want to see a schedule where each task is estimated to take X days. It communicates that you've really thought through the plan. Then, in terms of results, what they really care about is that we track to the date we committed to for the overall effort. If a senior director is drilling down into a schedule and asking, "Why is this 5-day task taking Bob 7 days to finish?" that's a strong signal that it's time to change your management.

sidlls 6y ago

I’d argue the predictive power of story point methodology has not been established. There are simply too many variables to make estimates more accurate than “best guess”: 1/ the team’s composition at a point in time (and all the variability that comes with this) 2/ the team’s dependencies (a kind of exponential effect is at work here: the team’s dependencies also have all these issues in variability and so do their dependencies) 3/ the type of work being estimated in the context of the product (even “basic” things like “CRUD” can be quite intricate in some cases) Basically unless the estimation is literally for a known change (e.g. modify source code files X, Y and Z with code changes A, B, and C) its predictive power diminishes substantially. It’s not a powerful tool, it’s a micromanagement technique disguised as a tool to empower ICs.

steelframe 6y ago
I completely agree with the impact of the variables you listed. However it's often the case that you need a starting point to estimate timelines. It's never acceptable to tell the decision makers, "We really have no idea how long this will take." Nor is it acceptable to say, "Sure, we'll have it ready in <licked finger in the wind> 8 weeks! Everyone get to work and make that happen (or else)!" Having a concrete plan ("micromanagement?") for getting to the finish line is important, and the story point methodology is an intuitive tool for figuring out what lengths the tasks need to start out with on the timeline.
- sidlls 6y ago
  Having a concrete plan with estimates is important. Assigning some "point value" to them, which variously means "complexity", "man-day-equivalent", "man-sprint-equivalent", and possibly others depending on (arbitrarily applied) context and individuals involved, is the gateway to micromanagement. It's entirely possible to provide meaningful estimates of both complexity and time-to-implement without resorting to story point formulations or any particular PM framework. "This will take about 6 weeks and has a few dependencies and unknowns that make it moderately risky to take longer." Rephrasing that as "This will take 3 sprints and has a story point value of 8 points per print" doesn't add any value, except from management's perspective as a way to put some "objective" quantification to the problem of estimates and deliverables.

jniedrauer 6y ago

Have story points been useful to you, even when incentives are aligned? I find that they're often orders of magnitude off, in either direction.

steelframe 6y ago
My team has been tracking its story-point-based velocity for about 20 months. The team's composition, product, and structure has been largely unchanged. We started tracking when we were newly formed, and we continued to use the same methodology until we shipped GA. When not focused on just one objective and not under deadline pressure, the team's average story point velocity per week was 30. This was when we tracked anything and everything we did. When we've track against a launch, where we only focused on the minimum set of tasks we needed to meet a clearly-defined objective with inter-team dependencies and a deadline, the team's average velocity has been 12. We don't use story points alone in forecasting timelines. What we do is presume a per-dev weekly velocity of (12 / team size). With all of the tasks on a chart, we work with the team to assign tasks to individuals, and we create a strawman schedule for how all the work gets done to meet the timeline. The "length" of each bug is a function of (12 / time size). We track and reassess how all the tasks are coming along on a weekly basis, with the understanding that the "strawman" schedule will end up looking different than we initially project. For example, take a team size of 6. Then the story points per week per dev is (12 / 6 = 2). Story points per weekday is (2 / 5). So for a task of size 3, we expect one dev to take ((5 / 2) * 3 = 7.5) days to complete, and so the length of the task on the timeline is (rounded up) 8 days. Then we bring the "fudge factors" into play. For the devs who are more productive we tend to schedule them back-to-back with the tasks that are more urgent and/or are blockers. For the devs that tend to need more time, we keep a lot of whitespace between the end of their assigned tasks and the deadline. The only time this methodology has gotten us into trouble is when we've caved to pressure by upper management to say we can actually do 15 story points per week or whatever. The most important thing is to measure actual velocity under as similar conditions as you can, and then refuse to accept any projection with a velocity other than you've actually observed. Story points are opaque outside of your team, and upper management is usually satisfied when you translate the story points into the schedule using the method I described.

Fire-Dragon-DoL 6y ago

What about replacing story points with time ranges? That gives you both time (min, max, which is the information you need) as well as the risk (distance between min and max). Then countermeasures could be taken to reduce the risk, until min and max are reasonably close. Sometimes max is +infinity when there is 0 knowledge, but even that information is useful: missing a role or need research.

steelframe 6y ago
My team uses story points as an input into a time range function.
- Fire-Dragon-DoL 6y ago
  How does it split the time from the risk? My main problem with points is the impossibility to represent: - A long task of 5 days that's very well known, it's 5 days of work - A short task that's very uncertain. It could be a one line change or many days of work With time range, you can represent that and it's very visible. With points, you can represent that as the same points, but that does not reflect reality, one of the two estimates is potentially wrong. This also helps providing input on what research tasks should be performed and what's their value.
  - steelframe 6y ago
    My team uses story points to estimate large projects over long periods of time. Sometimes things are overestimated, and other times things are underestimated. However, when you sum them all together, the errors tend to cancel each other out. I've found that upper management doesn't care whether any given task actually takes 3 days when we estimate it at 3 days. They just want to see a schedule where each task is estimated to take X days. It communicates that you've really thought through the plan. Then, in terms of results, what they really care about is that we track to the date we committed to for the overall effort. If a senior director is drilling down into a schedule and asking, "Why is this 5-day task taking Bob 7 days to finish?" that's a strong signal that it's time to change your management.