3 ms·

The law of large numbers doesn't help here because a) the error distribution is strongly one-sided (it being much, much easier for something to take twice as lo

The law of large numbers doesn't help here because a) the error distribution is strongly one-sided (it being much, much easier for something to take twice as long than half as long) and b) there aren't enough projects, typically, for implementation teams to stay stable for long enough for the central limit theorem to compensate. I wouldn't be at all surprised if the relationship between estimate and actual typically followed a power law relationship, in which case speaking about any averaging over time is meaningless anyway.

MakeUsersWant 3y ago
I've heard it's lognormal, which doesn't follow the central limit theorem either.
ohthehugemanate 3y ago
Great response. Part of the point of estimating difficulty is exactly to reduce the one-sidedness evident in time estimations. Human brains are much more accurate and consistent with effort estimation than time. It's still very likely a beta distribution but for the level of accuracy we're talking about it's just fine. We should probably clarify the "bar" here: the very best time estimators never beat about +30% delta from actual time. (Note that I'm not using +/- intervals because it's only ever +). So how many points (effort/time ratio values) does it take for convergence to get your effort/time ratio mean within 30% of the peak of the curve?
- regularfry 3y ago
  I don't think that 30% bar is commonly understood. From details in conversations I've had in the past where the reaction to (for instance) a 50% miss was "we can't ever be that bad at estimating again", there's a pervasive attitude that it's a fault in implementation teams that they're not accurate to the day. There's no middle ground: a team that can't give "good enough" estimates is held as incompetent, while the best anyone can ever do would be unacceptable. What's interesting on that specific 50% miss I saw is that the original estimate from the actual teams themselves was argued down as "too big, do it again" by their own management...

MakeUsersWant 3y ago

I've heard it's lognormal, which doesn't follow the central limit theorem either.

ohthehugemanate 3y ago

Great response. Part of the point of estimating difficulty is exactly to reduce the one-sidedness evident in time estimations. Human brains are much more accurate and consistent with effort estimation than time. It's still very likely a beta distribution but for the level of accuracy we're talking about it's just fine. We should probably clarify the "bar" here: the very best time estimators never beat about +30% delta from actual time. (Note that I'm not using +/- intervals because it's only ever +). So how many points (effort/time ratio values) does it take for convergence to get your effort/time ratio mean within 30% of the peak of the curve?

regularfry 3y ago
I don't think that 30% bar is commonly understood. From details in conversations I've had in the past where the reaction to (for instance) a 50% miss was "we can't ever be that bad at estimating again", there's a pervasive attitude that it's a fault in implementation teams that they're not accurate to the day. There's no middle ground: a team that can't give "good enough" estimates is held as incompetent, while the best anyone can ever do would be unacceptable. What's interesting on that specific 50% miss I saw is that the original estimate from the actual teams themselves was argued down as "too big, do it again" by their own management...