4 ms·

>The intelligence analysts who ultimately located Osama bin Laden worked on the problem for years. If measured at any point, the productivity of those analysts

>The intelligence analysts who ultimately located Osama bin Laden worked on the problem for years. If measured at any point, the productivity of those analysts would have been zero. Month after month, their failure rate was 100 per cent, until they achieved success. I guess if the metric was "caught Osama Bin Laden or not", then yeah, I agree. But who on earth would measure the analyst's productivity just so? Nothing but a strawman. >The source of the trouble is that when people are judged by performance metrics they are incentivised to do what the metrics measure, and what the metrics measure will be some established goal. But that impedes innovation, which means doing something not yet established, indeed that hasn’t even been tried out. Why? If a (admittedly weak) metric like "# of paying customers" is used as a goal, how does that "impede innovation"? How are people being stopped from coming up with creative ways to get more customers? Why do we assume the ways in which they improve the metric will be bad, and not innovative?

jimktrains2 8y ago
> I guess if the metric was "caught Osama Bin Laden or not", then yeah, I agree. > But who on earth would measure the analyst's productivity just so? Nothing but a strawman. Is that much different than measuring LoC, tickets closed, or features implemented? Some features just take time -- sometimes just the research alone to implement something could take a few weeks where literally nothing measurable happens. That's the point: completion metrics aren't useful for larger or open-ended projects. There was no gnatt chart, no schedule for finding Osama bin Laden; similarly you sometimes cannot plan out a large software project from the very start.
snovv_crash 8y ago
Because it doesn't take inyo account CAC, which might be higher than the profits made per customer.
tehjoker 8y ago
I really disliked that example. The Pakistanis knew where he was for years and the CIA sent teams of "vaccinators" to conduct a hunt for him, making many people in need of vaccinations distrust modern medicine, a crime that no one will answer for that could easily cost more lives than bin Laden ever did. This is OT, but if you're interested in understanding this epochal event in American media narrative, there's no one better than Sy Hersh to listen to: https://www.lrb.co.uk/v37/n10/seymour-m-hersh/the-killing-of-osama-bin-laden https://www.lrb.co.uk/v37/n10/seymour-m-hersh/the-killing-of... In a way, I guess it is on topic because it shows how "metric fixation" leads to unjust and dangerous outcomes for the people of the world.

jimktrains2 8y ago

> I guess if the metric was "caught Osama Bin Laden or not", then yeah, I agree. > But who on earth would measure the analyst's productivity just so? Nothing but a strawman. Is that much different than measuring LoC, tickets closed, or features implemented? Some features just take time -- sometimes just the research alone to implement something could take a few weeks where literally nothing measurable happens. That's the point: completion metrics aren't useful for larger or open-ended projects. There was no gnatt chart, no schedule for finding Osama bin Laden; similarly you sometimes cannot plan out a large software project from the very start.

snovv_crash 8y ago

Because it doesn't take inyo account CAC, which might be higher than the profits made per customer.

tehjoker 8y ago

I really disliked that example. The Pakistanis knew where he was for years and the CIA sent teams of "vaccinators" to conduct a hunt for him, making many people in need of vaccinations distrust modern medicine, a crime that no one will answer for that could easily cost more lives than bin Laden ever did. This is OT, but if you're interested in understanding this epochal event in American media narrative, there's no one better than Sy Hersh to listen to: https://www.lrb.co.uk/v37/n10/seymour-m-hersh/the-killing-of-osama-bin-laden https://www.lrb.co.uk/v37/n10/seymour-m-hersh/the-killing-of... In a way, I guess it is on topic because it shows how "metric fixation" leads to unjust and dangerous outcomes for the people of the world.