6 ms·

Reward individuals for getting things done. Not itself an easy task, but as long as everyone makes roughly the same salary... and as long as those jobs are gate

Reward individuals for getting things done. Not itself an easy task, but as long as everyone makes roughly the same salary... and as long as those jobs are gated in the first place by credentials and ridiculous interviews... Most everywhere I see progress, there's some kind of figure of merit or leaderboard. Yes, optimizing to a metric leads to problems, but it's better than optimizing to nothing. And those problems can be mitigated https://twitter.com/clumma/status/1329862461151997952 https://twitter.com/clumma/status/1329862461151997952

lmm 4y ago
> Yes, optimizing to a metric leads to problems, but it's better than optimizing to nothing. And those problems can be mitigated Citation needed. Elsewhere I've seen it claimed that actually, most people want to do a good job, and so any metric ends up doing more harm than good. And half of the suggestions in that twitter thread (keeping the metric private, changing it over time) seem to point in the direction of actually not having a metric and relying on human judgement instead.
- Timwi 4y ago
  > Elsewhere I've seen it claimed that actually, most people want to do a good job, and so any metric ends up doing more harm than good. Indeed. This is Goodhart’s Law.
  - lmm 4y ago
    Goodhart's Law is that the metric becomes an invalid measure. What I find interesting is the stronger claim, that performance on the thing you were actually trying to encourage will go down.
    - beckingz 4y ago
      Invalidating the metric after it becomes a target means that the metric is no longer an indicator of performance, so you have no guarantees that an increase in the metric results in an increase in performance.
      - Groxx 4y ago
        In some ways I think it follows trivially from the law: energy has been redirected away from the-thing-you-want and towards the-thing-you-measure. Unless you think effort spent on the metric increases the total energy of the system, anyway. Personally I would bet it pretty much always decreases it, not even breaking even.
deleted 4y ago
[deleted]

lmm 4y ago

> Yes, optimizing to a metric leads to problems, but it's better than optimizing to nothing. And those problems can be mitigated Citation needed. Elsewhere I've seen it claimed that actually, most people want to do a good job, and so any metric ends up doing more harm than good. And half of the suggestions in that twitter thread (keeping the metric private, changing it over time) seem to point in the direction of actually not having a metric and relying on human judgement instead.

Timwi 4y ago
> Elsewhere I've seen it claimed that actually, most people want to do a good job, and so any metric ends up doing more harm than good. Indeed. This is Goodhart’s Law.
- lmm 4y ago
  Goodhart's Law is that the metric becomes an invalid measure. What I find interesting is the stronger claim, that performance on the thing you were actually trying to encourage will go down.
  - beckingz 4y ago
    Invalidating the metric after it becomes a target means that the metric is no longer an indicator of performance, so you have no guarantees that an increase in the metric results in an increase in performance.
    - Groxx 4y ago
      In some ways I think it follows trivially from the law: energy has been redirected away from the-thing-you-want and towards the-thing-you-measure. Unless you think effort spent on the metric increases the total energy of the system, anyway. Personally I would bet it pretty much always decreases it, not even breaking even.

deleted 4y ago

[deleted]