7 ms·

It's not just the je ne sais quoi performance aspect that I'd be concerned about, it's something I'd call Wii-Sports-ification: the first time you try out movem

by t_mann 3y ago

It's not just the je ne sais quoi performance aspect that I'd be concerned about, it's something I'd call Wii-Sports-ification: the first time you try out movement-controlled games like they made the Wii console popular, you'll likely put a lot of effort into the movements as if you were wielding a real tennis racket or a golf club. After a while, you realize that the motion sensor is much more coarse, and you do much more efficient, bastardized versions of the original movements. While I acknowledge that this system is supposed to be much more sensitive, I'd expect a similar effect to be happening once athletes get their hands on those machines in training: they'll figure out how to best trigger the detection of each move, and how much flexing of the toes, bending in the back and overall effort is just enough for the machine, and it's going to change the look and feel of the sport.

steveBK123 3y ago
Exactly, its basically a variation of Goodhart's Law - “when a measure becomes a target, it ceases to be a good measure.”
- Nashooo 3y ago
  Not really. It depends on the quality of the measure and any underlying assumptions. If the target is knowing who is the fastest then a simple stopwatch (measure) is 100% aligned with the target. So the measure becoming the target is completely fine. The problem comes when one (or multiple) measure(s) are an approximation of some ideal measure for the target. For example the famous example [1] of the measuring rat tails to determine killed rats (target). The faulty assumption here was that the only way to hand in a rat's tail is by killing them. [1] https://en.wikipedia.org/wiki/Great_Hanoi_Rat_Massacre https://en.wikipedia.org/wiki/Great_Hanoi_Rat_Massacre
  - je42 3y ago
    Unfortunately not true. It is not a single metric. we have extra rules on top: No doping, separation of genders,...
  - IanCal 3y ago
    > Not really. It depends on the quality of the measure and any underlying assumptions. If the target is knowing who is the fastest then a simple stopwatch (measure) is 100% aligned with the target. So the measure becoming the target is completely fine. I get your point but with this case the measure is the target, so at no point does the "measurement become the target", it's aligned from the start. But then again, this is only true if your measure is of people performing the task you actually care about. If you want to know the fastest person and you measure over 100m you may end up with Bolt, which is great if you need "the fastest over 100m in a sprinting setup" but not if you need the fastest over a full marathon. In general, it's an extremely good first approximation - many measures result in poor targets, and care needs to be taken when turning a measure into a target. Even then it usually becomes a worse measure. > The faulty assumption here was that the only way to hand in a rat's tail is by killing them. More than that, it's an assumption that an increase in rat tails equals a decrease in live rats. Even if you kill a rat to get a rat tail, that doesn't necessarily hold true. It rewards those that breed 5 rats, release 4 and hand in the tail of 1. Killed rats would have still been a bad target. The true target was fewer live rats. Dead rats can be a reasonable measure, but as a target it has very risky outcomes.
  - quench 3y ago
    >Not really. It depends on the quality of the measure and any underlying assumptions. If the target is knowing who is the fastest then a simple stopwatch (measure) is 100% aligned with the target. So the measure becoming the target is completely fine. I am thinking of how runners all lean forward just before the finish line. This must slow their overall speed but gets a part of their body to the measurement datum slightly quicker. I admit its not a huge deviation between the measurement and the target but still it is an attempt to affect the measurement by means other than running fastest
  - rondini 3y ago
    I shouldn't be surprised given what website I'm on, but it's crazy that people will litigate the finer points of replacing human judgement of human arts rather than reject the premise outright.
    - steveBK123 3y ago
      Yes precisely. For reference my most AI/LLM pilled friends are also Desktop Linux enthusiasts, for whatever that's worth..
    - pixl97 3y ago
      Why is this crazy? Should we toss the entire field of science and go back to judging reality based on our feeling and perceptions?
      - psunavy03 3y ago
        Gymnastics is not a science. Judging "good" sports is purely an aesthetic choice.
        bluGill 3y ago
        There is a science to gymnastics. However the judging is only minimally based on the science and mostly subjective.
    - lupire 3y ago
      If something is wrong, there should be an articulable reason Find it.
  - lupire 3y ago
    That gets you people doing weird tricks to game the starting block and the finish line, which are artificial distractions from "fast running".
  - saghm 3y ago
    Only accepting entire rat bodies wouldn't necessarily work either; see the "cobra effect" that supposedly happened in colonial India: https://en.wikipedia.org/wiki/Perverse_incentive https://en.wikipedia.org/wiki/Perverse_incentive
- ta1243 3y ago
  There's a target of 9.8 seconds (or whatever) for the 100m sprint, there's a bunch of rules around things like windspeed, but it's still a good measure to find the "fastest man alive" despite being a target.
  - DRAGONERO 3y ago
    I think you mean “best shoe”
    - RcouF1uZ4gsC 3y ago
      You could wear the best most advanced shoes, and I bet you still couldn’t beat Usain Bolt wearing $10 sneakers.
      - jcalx 3y ago
        Two years post-retirement, wearing sweatpants and sneakers, he (albeit unofficially) tied the fastest 40-yard dash time at the NFL Combine: https://bleacherreport.com/articles/2818947-watch-usain-bolt-tie-nfl-combine-40-yard-dash-record-at-422-seconds https://bleacherreport.com/articles/2818947-watch-usain-bolt...
- cedilla 3y ago
  Isn't the measure already the target?
- kevinwang 3y ago
  Another suitable maxim here is "We shape our tools and then our tools shape us"
nottorp 3y ago
To summarize, before they were doing stuff that a human judge will notice. After, they will do stuff that the "AI" will notice. Goes for any physical sport that needs human judges instead of something measurable, like running speed, for their competitions. Pretty sure if you do competitional martial arts you train a subset of techniques that will be very visible and score you points as well.
- positr0n 3y ago
  A totally different but related example is pitch framing in baseball. Basically catchers will try to catch a pitch using a motion that slightly tricks the umpires eyes into calling a marginal ball a strike. It does have a statistically significant effect and since it has become widespread catchers are judged on how good they are at pitch framing since that is a small but significant part of the value to help the team win. That whole concept goes out the window if MLB ever changes to just using cameras to call balls and strikes!
- avery17 3y ago
  Yeah but the judges are human and are typically different humans at each event with different opinions and abilities to notice subtlties. There is no one standard human judge to abuse but with AI there is a single standard.
  - NeoTar 3y ago
    I wonder if having some intrinsic randomness to the AI model would help to mitigate this problem? So that each AI has a subtly different perception of the move, and in order to satisfy every possible judge, the gymnast would need to complete the move as comprehensively as possible.
    - badRNG 3y ago
      > I wonder if having some intrinsic randomness to the AI model would help to mitigate this problem? This would result in identical performances resulting in different scores with the difference being artificially arbitrary.
stcredzero 3y ago
I'd be concerned about, it's something I'd call Wii-Sports-ification AI might be able to radically change sports like fencing and HEMA. I could see the development of match judging AIs trained by experts. Anyone who has fenced knows the frustration of the fallibility of human judges. It's also possible that AI judging could benefit martial arts, bringing them closer to their practical roots, by counteracting the rule-gaming competition rules. ("Sport-ification" of martial arts.) Instead of allowing just any touch to count as a point, only hits with enough body structure to allow damaging force could be allowed. Many HEMA competitions try to implement this, but expert calibrated judging AIs would make this an order of magnitude better.
- o_nate 3y ago
  I was going to mention fencing. Thats a sport where there's a massive amount of judgment required from refs which is fairly opaque to non-specialists.
affinepplan 3y ago
as a former gymnast, I'm going to say I find this concern super unlikely
chongli 3y ago
they'll figure out how to best trigger the detection of each move, and how much flexing of the toes, bending in the back and overall effort is just enough for the machine, and it's going to change the look and feel of the sport. I remember watching Olympic diving years ago and the commentator was making a big deal out of the fact that the divers were spreading their knees apart (apparently a flaw) but that the judges weren't picking it up because their viewing angle on the dives didn't allow them to see it. So the divers were taking advantage of a literal blindspot in the human judging to make their dives easier to perform. So really, changing to AI judging is just going to change what the competitors optimize for in an arbitrary way. It's not going to fundamentally change things.
- Dalewyn 3y ago
  If anything, having computers do the judging with cameras setup from all directions with human judges available to override in particularly suspect rulings is more fair to everyone and everything involved: The players, the spectators, the judges, and the game. It will take away most of the fun(?) of arguing a judge's ruling and gaming them, but I will fundamentally always support making games fairer and objective'er.
  - ChatGTP 3y ago
    Right, we’re heading to a world where everything is “perfect”.
  - mywittyname 3y ago
    > more fair to everyone and everything involved I hope so. But I suspect a racial bias of some kind will be uncovered.
beambot 3y ago
At least with an AI-based system, it's presumably consistent across performances & devices and can be used during training. While not "objective" per se, it's certainly better than an human judge that is none of these things.
- dimitrios1 3y ago
  The concept of judging has always included some form of bias. There is no such thing as an impartial judge. AI won't change this, it will just include a different set of biases. I'd rather be judged by a human than a machine.
  - thfuran 3y ago
    >it will just include a different set of biases One that many people who would admit the possibility in a human judge will likely insist cannot be present.
- pfisherman 3y ago
  > Consistent across performances and devices LOL! I think the assumption that an ML system would be robust across these contexts is not well founded. Things as innocuous as the lighting, background, or placement of athletic tape could easily wreck a model like this. The thing about human judges is that the modes of failure are pretty well known at this point; and human perception and visual processing is pretty robust to perturbation.
deleted 3y ago
[deleted]
josefresco 3y ago
The people that play Wii sports are not the same people that play professional sports. While I agree AI will be used to optimize routines, I highly doubt it will digress to the "I can play Wii tennis while laying down and only flicking my wrist" level. Nor do I think the "AI" sensors being used are as crude as a cheap, consumer focused Wii remote released over 15 years ago.
- rurp 3y ago
  > The people that play Wii sports are not the same people that play professional sports. Right, professional athletes will go to much greater lengths to exploit a scoring sytem if they can get an edge from it. The AI sensors might be much better than a Wii, but that won't fix the core problem because they will be studied and gamed much more diligently.
  - thfuran 3y ago
    I, for one, would quite like to see what sort of bizarre adversarial flailing technically counts as a perfect performance. If we can get one totally surreal Olympics out of it, I'd call it a win.
    - mywittyname 3y ago
      Watch Olympic Sabre fencing. There's all kinds of wild flailing that people do in that sport. All that really counts is that the hit registers. So bunny hopping towards your opponent flailing that sabre wildly is a perfectly valid tactic.

by t_mann 3y ago

steveBK123 3y ago

Exactly, its basically a variation of Goodhart's Law - “when a measure becomes a target, it ceases to be a good measure.”

Nashooo 3y ago
Not really. It depends on the quality of the measure and any underlying assumptions. If the target is knowing who is the fastest then a simple stopwatch (measure) is 100% aligned with the target. So the measure becoming the target is completely fine. The problem comes when one (or multiple) measure(s) are an approximation of some ideal measure for the target. For example the famous example [1] of the measuring rat tails to determine killed rats (target). The faulty assumption here was that the only way to hand in a rat's tail is by killing them. [1] https://en.wikipedia.org/wiki/Great_Hanoi_Rat_Massacre https://en.wikipedia.org/wiki/Great_Hanoi_Rat_Massacre
- je42 3y ago
  Unfortunately not true. It is not a single metric. we have extra rules on top: No doping, separation of genders,...
- IanCal 3y ago
  > Not really. It depends on the quality of the measure and any underlying assumptions. If the target is knowing who is the fastest then a simple stopwatch (measure) is 100% aligned with the target. So the measure becoming the target is completely fine. I get your point but with this case the measure is the target, so at no point does the "measurement become the target", it's aligned from the start. But then again, this is only true if your measure is of people performing the task you actually care about. If you want to know the fastest person and you measure over 100m you may end up with Bolt, which is great if you need "the fastest over 100m in a sprinting setup" but not if you need the fastest over a full marathon. In general, it's an extremely good first approximation - many measures result in poor targets, and care needs to be taken when turning a measure into a target. Even then it usually becomes a worse measure. > The faulty assumption here was that the only way to hand in a rat's tail is by killing them. More than that, it's an assumption that an increase in rat tails equals a decrease in live rats. Even if you kill a rat to get a rat tail, that doesn't necessarily hold true. It rewards those that breed 5 rats, release 4 and hand in the tail of 1. Killed rats would have still been a bad target. The true target was fewer live rats. Dead rats can be a reasonable measure, but as a target it has very risky outcomes.
- quench 3y ago
  >Not really. It depends on the quality of the measure and any underlying assumptions. If the target is knowing who is the fastest then a simple stopwatch (measure) is 100% aligned with the target. So the measure becoming the target is completely fine. I am thinking of how runners all lean forward just before the finish line. This must slow their overall speed but gets a part of their body to the measurement datum slightly quicker. I admit its not a huge deviation between the measurement and the target but still it is an attempt to affect the measurement by means other than running fastest
- rondini 3y ago
  I shouldn't be surprised given what website I'm on, but it's crazy that people will litigate the finer points of replacing human judgement of human arts rather than reject the premise outright.
  - steveBK123 3y ago
    Yes precisely. For reference my most AI/LLM pilled friends are also Desktop Linux enthusiasts, for whatever that's worth..
  - pixl97 3y ago
    Why is this crazy? Should we toss the entire field of science and go back to judging reality based on our feeling and perceptions?
    - psunavy03 3y ago
      Gymnastics is not a science. Judging "good" sports is purely an aesthetic choice.
      - bluGill 3y ago
        There is a science to gymnastics. However the judging is only minimally based on the science and mostly subjective.
  - lupire 3y ago
    If something is wrong, there should be an articulable reason Find it.
- lupire 3y ago
  That gets you people doing weird tricks to game the starting block and the finish line, which are artificial distractions from "fast running".
- saghm 3y ago
  Only accepting entire rat bodies wouldn't necessarily work either; see the "cobra effect" that supposedly happened in colonial India: https://en.wikipedia.org/wiki/Perverse_incentive https://en.wikipedia.org/wiki/Perverse_incentive
ta1243 3y ago
There's a target of 9.8 seconds (or whatever) for the 100m sprint, there's a bunch of rules around things like windspeed, but it's still a good measure to find the "fastest man alive" despite being a target.
- DRAGONERO 3y ago
  I think you mean “best shoe”
  - RcouF1uZ4gsC 3y ago
    You could wear the best most advanced shoes, and I bet you still couldn’t beat Usain Bolt wearing $10 sneakers.
    - jcalx 3y ago
      Two years post-retirement, wearing sweatpants and sneakers, he (albeit unofficially) tied the fastest 40-yard dash time at the NFL Combine: https://bleacherreport.com/articles/2818947-watch-usain-bolt-tie-nfl-combine-40-yard-dash-record-at-422-seconds https://bleacherreport.com/articles/2818947-watch-usain-bolt...
cedilla 3y ago
Isn't the measure already the target?
kevinwang 3y ago
Another suitable maxim here is "We shape our tools and then our tools shape us"

nottorp 3y ago

To summarize, before they were doing stuff that a human judge will notice. After, they will do stuff that the "AI" will notice. Goes for any physical sport that needs human judges instead of something measurable, like running speed, for their competitions. Pretty sure if you do competitional martial arts you train a subset of techniques that will be very visible and score you points as well.

positr0n 3y ago
A totally different but related example is pitch framing in baseball. Basically catchers will try to catch a pitch using a motion that slightly tricks the umpires eyes into calling a marginal ball a strike. It does have a statistically significant effect and since it has become widespread catchers are judged on how good they are at pitch framing since that is a small but significant part of the value to help the team win. That whole concept goes out the window if MLB ever changes to just using cameras to call balls and strikes!
avery17 3y ago
Yeah but the judges are human and are typically different humans at each event with different opinions and abilities to notice subtlties. There is no one standard human judge to abuse but with AI there is a single standard.
- NeoTar 3y ago
  I wonder if having some intrinsic randomness to the AI model would help to mitigate this problem? So that each AI has a subtly different perception of the move, and in order to satisfy every possible judge, the gymnast would need to complete the move as comprehensively as possible.
  - badRNG 3y ago
    > I wonder if having some intrinsic randomness to the AI model would help to mitigate this problem? This would result in identical performances resulting in different scores with the difference being artificially arbitrary.

stcredzero 3y ago

I'd be concerned about, it's something I'd call Wii-Sports-ification AI might be able to radically change sports like fencing and HEMA. I could see the development of match judging AIs trained by experts. Anyone who has fenced knows the frustration of the fallibility of human judges. It's also possible that AI judging could benefit martial arts, bringing them closer to their practical roots, by counteracting the rule-gaming competition rules. ("Sport-ification" of martial arts.) Instead of allowing just any touch to count as a point, only hits with enough body structure to allow damaging force could be allowed. Many HEMA competitions try to implement this, but expert calibrated judging AIs would make this an order of magnitude better.

o_nate 3y ago
I was going to mention fencing. Thats a sport where there's a massive amount of judgment required from refs which is fairly opaque to non-specialists.

affinepplan 3y ago

as a former gymnast, I'm going to say I find this concern super unlikely

chongli 3y ago

they'll figure out how to best trigger the detection of each move, and how much flexing of the toes, bending in the back and overall effort is just enough for the machine, and it's going to change the look and feel of the sport. I remember watching Olympic diving years ago and the commentator was making a big deal out of the fact that the divers were spreading their knees apart (apparently a flaw) but that the judges weren't picking it up because their viewing angle on the dives didn't allow them to see it. So the divers were taking advantage of a literal blindspot in the human judging to make their dives easier to perform. So really, changing to AI judging is just going to change what the competitors optimize for in an arbitrary way. It's not going to fundamentally change things.

Dalewyn 3y ago
If anything, having computers do the judging with cameras setup from all directions with human judges available to override in particularly suspect rulings is more fair to everyone and everything involved: The players, the spectators, the judges, and the game. It will take away most of the fun(?) of arguing a judge's ruling and gaming them, but I will fundamentally always support making games fairer and objective'er.
- ChatGTP 3y ago
  Right, we’re heading to a world where everything is “perfect”.
- mywittyname 3y ago
  > more fair to everyone and everything involved I hope so. But I suspect a racial bias of some kind will be uncovered.

beambot 3y ago

At least with an AI-based system, it's presumably consistent across performances & devices and can be used during training. While not "objective" per se, it's certainly better than an human judge that is none of these things.

dimitrios1 3y ago
The concept of judging has always included some form of bias. There is no such thing as an impartial judge. AI won't change this, it will just include a different set of biases. I'd rather be judged by a human than a machine.
- thfuran 3y ago
  >it will just include a different set of biases One that many people who would admit the possibility in a human judge will likely insist cannot be present.
pfisherman 3y ago
> Consistent across performances and devices LOL! I think the assumption that an ML system would be robust across these contexts is not well founded. Things as innocuous as the lighting, background, or placement of athletic tape could easily wreck a model like this. The thing about human judges is that the modes of failure are pretty well known at this point; and human perception and visual processing is pretty robust to perturbation.

deleted 3y ago

[deleted]

josefresco 3y ago

The people that play Wii sports are not the same people that play professional sports. While I agree AI will be used to optimize routines, I highly doubt it will digress to the "I can play Wii tennis while laying down and only flicking my wrist" level. Nor do I think the "AI" sensors being used are as crude as a cheap, consumer focused Wii remote released over 15 years ago.

rurp 3y ago
> The people that play Wii sports are not the same people that play professional sports. Right, professional athletes will go to much greater lengths to exploit a scoring sytem if they can get an edge from it. The AI sensors might be much better than a Wii, but that won't fix the core problem because they will be studied and gamed much more diligently.
- thfuran 3y ago
  I, for one, would quite like to see what sort of bizarre adversarial flailing technically counts as a perfect performance. If we can get one totally surreal Olympics out of it, I'd call it a win.
  - mywittyname 3y ago
    Watch Olympic Sabre fencing. There's all kinds of wild flailing that people do in that sport. All that really counts is that the hit registers. So bunny hopping towards your opponent flailing that sabre wildly is a perfectly valid tactic.