5 ms·

"...when you’re incentivising organisational behaviour with metrics, there are really only three ways people might respond: 1) they might improve the system, 2)

"...when you’re incentivising organisational behaviour with metrics, there are really only three ways people might respond: 1) they might improve the system, 2) they might distort the system, or 3) they might distort the data." This is wrong, and the wrongness of it undermines the whole piece, I think: - A fourth way people respond is to oppose the choice of target and/or metric; to question its value and lobby to change it. - A fifth way people respond is to oppose the whole idea of incentives on the basis of metrics (perhaps by citing Goodhart's Law... which is a use of Goodhart's Law). Goodhart's Law is useful not just because it reminds us that making a metric a target may incentivize behavior that makes THAT metric a poor indicator of a good system, but also because choosing ANY metric as a target changes everyone's relationship with ALL metrics-- it spells the end of inquiry and the beginning of what might be called compliance anxiety.

mkleczek 2y ago
How can people (either individuals or groups) assess (or even define) achieving goals without SOME metric?
Lvl999Noob 2y ago
While true, I think your additions to the behaviors are rather... useless. Out of the original three, notice that one is the actual behaviour we want to happen and two are insidious side effects that we want to prevent. Your proposed fourth and fifth outcome behaviours, on the other hand, are neither. Most importantly, they are transient (at least ideally). Either the workforce and the management come to an agreement and metrics continue (or discontinue) or they don't and the business stays in a limbo. It is an emergency (or some other word with lower impact; incident?). There isn't a covert resistance by some teams specifically working against the metric and lowering it while also hiding themselves from notice.
- satisfice 2y ago
  My additions are important if you want to understand the value of Goodhart’s Law. They are logically necessary for me to make my point— which is that his analysis of the situation is flawed. I am bemused that you deride them, given that they are, in fact, how I have responded to metrics in technical projects since I first developed a metrics program for Borland, in ‘93. (I championed inquiry metrics and opposed control metrics.)
  - cutemonster 2y ago
    > inquiry metrics That sounds interesting, how did you do that? What different inquiry metrics did you use if I can ask, what did the others think, how did it all work out?

mkleczek 2y ago

How can people (either individuals or groups) assess (or even define) achieving goals without SOME metric?

Lvl999Noob 2y ago

While true, I think your additions to the behaviors are rather... useless. Out of the original three, notice that one is the actual behaviour we want to happen and two are insidious side effects that we want to prevent. Your proposed fourth and fifth outcome behaviours, on the other hand, are neither. Most importantly, they are transient (at least ideally). Either the workforce and the management come to an agreement and metrics continue (or discontinue) or they don't and the business stays in a limbo. It is an emergency (or some other word with lower impact; incident?). There isn't a covert resistance by some teams specifically working against the metric and lowering it while also hiding themselves from notice.

satisfice 2y ago
My additions are important if you want to understand the value of Goodhart’s Law. They are logically necessary for me to make my point— which is that his analysis of the situation is flawed. I am bemused that you deride them, given that they are, in fact, how I have responded to metrics in technical projects since I first developed a metrics program for Borland, in ‘93. (I championed inquiry metrics and opposed control metrics.)
- cutemonster 2y ago
  > inquiry metrics That sounds interesting, how did you do that? What different inquiry metrics did you use if I can ask, what did the others think, how did it all work out?