4 ms·

Yeah, that's kinda expected. But what I'm more worried about in sports is when metrics(rules) become engrained in that sports culture and start shaping the spor

by divan 3y ago

Yeah, that's kinda expected. But what I'm more worried about in sports is when metrics(rules) become engrained in that sports culture and start shaping the sport itself. Like, if an athletics high-jump champion has to jump over a fence with concrete ground – would s/he be still the best in the world? Probably not, because they're focusing on the Fosberry flop technique which works only in a competition setting with a soft landing mat. Or in figure skating, where officials think that rules should reward "complex elements" and now everyone is chasing "complex" elements (quadruple jumps), skipping the "easy" elements (actual skating skills). One of the problem with a metric substituting the original True Goal is also when this True Goal is not defined (or different people defined it differently). This type of Goodhart's law is hard to fix. I'm not sure how many sports really have shared understanding of what's a true goal in their sports.

cafard 3y ago
As far as I know, neither the high jump nor the pole vault ever had hard landing areas. I'm sure that foam rubber is a lot better than dried cow manure, but guys were still going way over 6' in the jump and 15' in the vault before foam rubber came along.
- divan 3y ago
  Right, but I can come up with an infinite number of examples. Take javelin throw – the essential skill is, well, throwing a javelin. There are many things that can theoretically be measured – distance, height, speed, jiggling, precision, throwing efficiency. For some reasons (historical, practical) distance was chosen. So now javelin throwers are good at distances but not at anything else. Like it's obvious that if you don't train to hit the target with a javelin, you are probably not going to be good at it. So it's unlikely our javelin throw champion can outperform the average ancient human in hunting mammoths, for example. I see "mastering javelin throwing skill" as a True Goal and "distance" as a metric. True Goal probably includes many aspects of the skill (let's say "distance" and "precision"), but as we measured only distance, others got neglected. The end result is far from True Goal because we disincentivize athletes to practice other aspects of that basic athletic skill. If the correlation between "being a truly great javelin thrower" (i.e. great at all aspects of the skill) and a "distance" existed, using it for competition resulted in the collapse of the correlation. Classic Goodhart's effect.
  - jacquesm 3y ago
    There are lots of real world examples of this same phenomenon and they usually result in ridiculous losses combined with minor gains on one parameter. Balance is the key but we tend to reward the specialists (records, finances). People always want to know who is 'best' at something. Triathlon is one attempt to break this but even there it is all quite physical. Chessboxing is another :)
  - hgsgm 3y ago
    Mammoths travel in herd, so accuracy isn't very important.
a_c 3y ago
I see it as a principal agent problem [0]. The agent being the sport players working according the game rules to maximize their gains. It could be pouring countless hours into mastering the sport, studying the rules to come up with best strategies [1], deliberately losing (e.g. for bribery), among others. The principal(s) are the perceived sportsmanship, while ill defined, of a particular sports It also happens to software engineering. The value of your code is solely depends on how many times it is being willingly run. Be it people using it happily or another piece of code is calling it without caveats. But we come up with all sorts of metrics, DORA, SPACE, developer experience, what not. For most family/small business, like restaurants, groceries, builders, etc, the only thing that matters are customers willing to return. Somehow people think that by mimicking what big companies' management practice does can makes you one of the big companies. While the most important thing, making things that people want, is grossly overlooked. People used to make fun of MBAs parachuting into management right out of school. That trademark is certainly open to all titles now. [0] https://en.wikipedia.org/wiki/Principal%E2%80%93agent_problem https://en.wikipedia.org/wiki/Principal%E2%80%93agent_proble... [1] https://www.youtube.com/watch?v=yh5c3duQQ1w https://www.youtube.com/watch?v=yh5c3duQQ1w

by divan 3y ago

cafard 3y ago

As far as I know, neither the high jump nor the pole vault ever had hard landing areas. I'm sure that foam rubber is a lot better than dried cow manure, but guys were still going way over 6' in the jump and 15' in the vault before foam rubber came along.

divan 3y ago
Right, but I can come up with an infinite number of examples. Take javelin throw – the essential skill is, well, throwing a javelin. There are many things that can theoretically be measured – distance, height, speed, jiggling, precision, throwing efficiency. For some reasons (historical, practical) distance was chosen. So now javelin throwers are good at distances but not at anything else. Like it's obvious that if you don't train to hit the target with a javelin, you are probably not going to be good at it. So it's unlikely our javelin throw champion can outperform the average ancient human in hunting mammoths, for example. I see "mastering javelin throwing skill" as a True Goal and "distance" as a metric. True Goal probably includes many aspects of the skill (let's say "distance" and "precision"), but as we measured only distance, others got neglected. The end result is far from True Goal because we disincentivize athletes to practice other aspects of that basic athletic skill. If the correlation between "being a truly great javelin thrower" (i.e. great at all aspects of the skill) and a "distance" existed, using it for competition resulted in the collapse of the correlation. Classic Goodhart's effect.
- jacquesm 3y ago
  There are lots of real world examples of this same phenomenon and they usually result in ridiculous losses combined with minor gains on one parameter. Balance is the key but we tend to reward the specialists (records, finances). People always want to know who is 'best' at something. Triathlon is one attempt to break this but even there it is all quite physical. Chessboxing is another :)
- hgsgm 3y ago
  Mammoths travel in herd, so accuracy isn't very important.

a_c 3y ago

I see it as a principal agent problem [0]. The agent being the sport players working according the game rules to maximize their gains. It could be pouring countless hours into mastering the sport, studying the rules to come up with best strategies [1], deliberately losing (e.g. for bribery), among others. The principal(s) are the perceived sportsmanship, while ill defined, of a particular sports It also happens to software engineering. The value of your code is solely depends on how many times it is being willingly run. Be it people using it happily or another piece of code is calling it without caveats. But we come up with all sorts of metrics, DORA, SPACE, developer experience, what not. For most family/small business, like restaurants, groceries, builders, etc, the only thing that matters are customers willing to return. Somehow people think that by mimicking what big companies' management practice does can makes you one of the big companies. While the most important thing, making things that people want, is grossly overlooked. People used to make fun of MBAs parachuting into management right out of school. That trademark is certainly open to all titles now. [0] https://en.wikipedia.org/wiki/Principal%E2%80%93agent_problem https://en.wikipedia.org/wiki/Principal%E2%80%93agent_proble... [1] https://www.youtube.com/watch?v=yh5c3duQQ1w https://www.youtube.com/watch?v=yh5c3duQQ1w