7 ms·

Sports is one of the most fascinating and underexplored examples of Goodhart's law (collapsing correlation between the metric and a goal). One of the most wide

• furyofantares 3y ago
  Also interesting is how lots of rules have standard penalties - and this turns them into potential strategic components rather than something you aren't allowed to do. It's not cheating to break these rules, it's considered part of the game.
  • dylan604 3y ago
    This is what is known as a "tactical foul" or taking one for the team
  • divan 3y ago
    Yeah, that's kinda expected. But what I'm more worried about in sports is when metrics(rules) become engrained in that sports culture and start shaping the sport itself. Like, if an athletics high-jump champion has to jump over a fence with concrete ground – would s/he be still the best in the world? Probably not, because they're focusing on the Fosberry flop technique which works only in a competition setting with a soft landing mat. Or in figure skating, where officials think that rules should reward "complex elements" and now everyone is chasing "complex" elements (quadruple jumps), skipping the "easy" elements (actual skating skills). One of the problem with a metric substituting the original True Goal is also when this True Goal is not defined (or different people defined it differently). This type of Goodhart's law is hard to fix. I'm not sure how many sports really have shared understanding of what's a true goal in their sports.
    • cafard 3y ago
      As far as I know, neither the high jump nor the pole vault ever had hard landing areas. I'm sure that foam rubber is a lot better than dried cow manure, but guys were still going way over 6' in the jump and 15' in the vault before foam rubber came along.
      • divan 3y ago
        Right, but I can come up with an infinite number of examples. Take javelin throw – the essential skill is, well, throwing a javelin. There are many things that can theoretically be measured – distance, height, speed, jiggling, precision, throwing efficiency. For some reasons (historical, practical) distance was chosen. So now javelin throwers are good at distances but not at anything else. Like it's obvious that if you don't train to hit the target with a javelin, you are probably not going to be good at it. So it's unlikely our javelin throw champion can outperform the average ancient human in hunting mammoths, for example. I see "mastering javelin throwing skill" as a True Goal and "distance" as a metric. True Goal probably includes many aspects of the skill (let's say "distance" and "precision"), but as we measured only distance, others got neglected. The end result is far from True Goal because we disincentivize athletes to practice other aspects of that basic athletic skill. If the correlation between "being a truly great javelin thrower" (i.e. great at all aspects of the skill) and a "distance" existed, using it for competition resulted in the collapse of the correlation. Classic Goodhart's effect.
        • jacquesm 3y ago
          There are lots of real world examples of this same phenomenon and they usually result in ridiculous losses combined with minor gains on one parameter. Balance is the key but we tend to reward the specialists (records, finances). People always want to know who is 'best' at something. Triathlon is one attempt to break this but even there it is all quite physical. Chessboxing is another :)
        • hgsgm 3y ago
          Mammoths travel in herd, so accuracy isn't very important.
  • kenjackson 3y ago
    I've always felt basketball is the worst with this of the major sports. It is a standard strategy to foul at the end of games when losing (or even if winning by 3 in the final seconds). The punishment for fouling is to allow the fouled player to get free throws, but free throws are a skill and not all players are skilled at it. So if you can foul the right person you can often change the probability of the game. A simple rule change could end this. For example, a foul in the last two minutes is a free throw and the ball.
    • kristjansson 3y ago
      Late game basketball fouls are good meta though. The strategic trade - a great chance at scoring, in exchange for the time the offense would have taken - is reasonable, and avoids the wasted last two-three minutes of American football. A team that’s down, but in contact isn’t totally dominated by the clock, but has the length of the game isn’t totally unbounded like baseball or tennis
  • jbandela1 3y ago
    Another example is pass interference in college football. It is a first down and 15 yards from the line of scrimmage. Thus if you are getting beat on a play, rather than potentially allowing 40 or 50 yards and a touchdown, it is better to blatantly pass interfere (tackle the receiver before they get the ball, etc).
    • cafard 3y ago
      Or, you might just manage it without the ref spotting it, as late in a recent Super Bowl.
  • pcchristie 3y ago
    A good example of that is Australian Rules Football. Egregious fouls or non-play acts like time wasting or violence used to get a 15m penalty (you just get to walk 15m down the ground unimpeded). But one coach realised that the time gained in awarding and jogging the 15m allowed his team to set up defence which was a worthwhile tradeoff and instructed his players to regularly refuse to give the ball back and concede these penalties when it was necessary. The league changed it to 50m to even it up. Funnily enough, in modern times the other rules have tightened so much that 50m is way overkill for most things it's awarded for.
• munificent 3y ago
  The meta-game of game design: how do you design a game whose mechanics lead to incentives that align with your intentions is a fascinating area. It also has deep connections to policy design in politics. In the political arena, it gets so much more complex because you have disagreement on intentions and people deliberately obfuscating their intentions. So you've got people arguing over legislation who are trying to aiming the laws have different emergent properties.
  • divan 3y ago
    Right. Do you have any good recommendations to read on this matter? Here are a couple of good papers on how metrics distort systems and what can be done to mitigate this. [1] [2]. And then a new post by Cedric Chin on practical examples of mitigating Goodhart's law in Amazon. Really good read: [3] But generally, I feel like not many people or organizations care about this. The non-profit sports world is conservative and slow, organizations and corporations are also often slow, and there some psychological/social effects linked to the metrics-that-don't-work-anymore. --- - [1] Categorizing Variants of Goodhart's Law - https://arxiv.org/abs/1803.04585 https://arxiv.org/abs/1803.04585 - [2] Building Less Flawed Metrics: Dodging Goodhart and Campbell's Laws - https://www.researchgate.net/publication/334478956_Building_Less_Flawed_Metrics_Dodging_Goodhart_and_Campbell%27s_Laws https://www.researchgate.net/publication/334478956_Building_... - [3] https://commoncog.com/goodharts-law-not-useful/ https://commoncog.com/goodharts-law-not-useful/
    • munificent 3y ago
      I don't but I'd love to read them if anyone else does.
    • sbierwagen 3y ago
      Spiritually related: examples of RL algorithms exploiting glitches in the physics engine (clipping into the ground to shoot forward at incredible speed) or gaming the goal state: (Agent that's supposed to learn how to walk instead evolving a body with a single long leg that just falls forward-- getting a high speed score with low complexity penalty) https://arxiv.org/abs/1803.03453 https://arxiv.org/abs/1803.03453 https://www.deepmind.com/blog/specification-gaming-the-flip-side-of-ai-ingenuity https://www.deepmind.com/blog/specification-gaming-the-flip-...
• soldarnal 3y ago
  The game clock is one of the more common rules that tends to warp gameplay in this way, with teams that are ahead trying to avoid play and run out the clock rather than continue to engage in the contest. The other team, meanwhile, resorts to increasingly desperate tactics like pulling the goalie, intentionally committing fouls, or laterals and trying for the onside kick. Related: https://en.wikipedia.org/wiki/Elam_Ending https://en.wikipedia.org/wiki/Elam_Ending
  • kodt 3y ago
    The end of American Football games is so frustrating because of this, with a team just constantly taking a knee to run out the clock. It makes the last 2 minutes of the game useless. Constant fouls in Basketball hoping the opposing team will miss a free throw is another frustrating tactic that drags the last 2 minutes on forever, in what is usually a forgone conclusion anyway.
    • hgsgm 3y ago
      You are allowed to go to the kitchen after 58minutes.
      • defrost 3y ago
        Although sometimes it's wise come back 50 minutes later for the very last minutes of a tight Grand Final. https://youtu.be/_qLF1qFn6u8?t=6477 https://youtu.be/_qLF1qFn6u8?t=6477 ( WTF is AFL: https://www.youtube.com/watch?v=u_SqfNNfhmM https://www.youtube.com/watch?v=u_SqfNNfhmM )
        • stevage 3y ago
          Wow that's a pretty heroic final mark!
    • freetime2 3y ago
      I find clock management to be an interesting aspect of the game in American Football, because literally every second counts. It's one of the things that separates elite players and coaches from everyone else. Tom Brady, for example, was renowned for his ability to move the ball down the field and score in the last two minutes of a game. Sure sometimes it means the game is effectively over 2 minutes early if the leading team gets possession and the trailing team has no timeouts remaining. But in that case it just has the effect of shifting the decisive moment a little bit earlier (if it's a close game). And the rare games that end with 0 seconds on the clock on a game winning touchdown or field goal are truly memorable.
• bombcar 3y ago
  You find this in online games quite a bit - the have a qualifying round in a tournament which divides players into ten divisions. The top three out of ten in the top four divisions get a great prize, and the top one in the remaining get a slightly lesser version. So it makes sense to smurf yourself into a lower division and hope to win the one prize than compete with equal skill players for the three prizes.
• NikolaNovak 3y ago
  There was a period in world rally championship where it was beneficial not to win day one or day two of a 3 day championship. This resulted in hilarious but rational situations where top racers would race like maniacs for the entire day, then 200meters before the end of the a last stage of the day slow down and crawl for say 12 seconds, so they would come a second behind somebody else and not lead the pack the next day (leading the pack effectively means clearing the path on gravel roads for the rest of the racers) It was meant to even the field by handicapping the front runners (the Mario kart approach I suppose). But it backfired spectacularly. I think they went back to randomization after that.
  • manquer 3y ago
    why drive like maniacs only to slow down at the end ? might as well drive slower throughout the stage ? and be most aggressive on day 3 ?
    • Pet_Ant 3y ago
      Because you wanted second place, not third or latter? Maybe because you didn’t want the track too chewed up?
      • manquer 3y ago
        You don’t know what is going to be second or any other place , every one is going to be doing same thing . You would only know what the current leader board looks like . Remember rally is raced against the clock not against other cars. If you had a target time you should be be able pace yourself reasonably close to that time ?
        • NikolaNovak 3y ago
          Depending on your placement, you might. I.e. If my biggest competitor is ahead of me in order, the team will radio me their split times and finish times and tell me exactly the finish time I need to be second behind them. Again, the system intentionally put champions up front, so everybody behind them knew the champions finish time. It was a self reinforcing system :-/