5 ms·

I learned a new term in the context of AI recently: "Specification Gaming". There's a big list here: https://t.co/OqoYN8MvMN https://t.co/OqoYN8MvMN But it's

I learned a new term in the context of AI recently: "Specification Gaming". There's a big list here: https://t.co/OqoYN8MvMN https://t.co/OqoYN8MvMN But it's stuff like: - Evolved algorithm for landing aircraft exploited overflow errors in the physics simulator by creating large forces that were estimated to be zero, resulting in a perfect score - A cooperative GAN architecture for converting images from one genre to another (eg horses<->zebras) has a loss function that rewards accurate reconstruction of images from its transformed version; CycleGAN turns out to partially solve the task by, in addition to the cross-domain analogies it learns, steganographically hiding autoencoder-style data about the original image invisibly inside the transformed image to assist the reconstruction of details. - Simulated pancake making robot learned to throw the pancake as high in the air as possible in order to maximize time away from the ground - Robot hand pretending to grasp an object by moving between the camera and the object - Self-driving car rewarded for speed learns to spin in circles All of which leads me to think that if you can't at some level explain how/what/why it's reaching a certain conclusion that it may be reaching a radically different end than you're anticipating.

neel8986 8y ago
Great examples. I think algorithms in sites like Facebook did something similar. In order to maximize view/clicks, they themselves created problems like an echo chamber or promoting divisive articles which in turn made the whole experience worse
- reitanqild 8y ago
  And Google has "simplified" search by fuzzing terms until it is somewhat possible to use even if you really cannat spel. In the process they broke it so badly I now prefer DDG. (Who also gives me results I didn't ask for.)
- sizzle 8y ago
  Facebook algorithms making the whole experience worse is being quite charitable in describing what it did, considering it's huge role in spreading of fake news and biasing people's thinking towards the presidential election along with creating strong negative feedback loops.
renjimen 8y ago
Not quite the same, but equally as disastrous is data leakage. This comes in some fun and unexpected forms. One example is from the fisheries monitoring competition on Kaggle, where NNs were learning to predict fish caught using the background image of the boats rather than the fish shown on camera. i.e. different boats tend to catch different fish
- devoply 8y ago
  inductive learning.
- thisisit 8y ago
  I have recently started learning ML and trying competitions on Kaggle. And I also seeing that in many of the competitions I have tried. The biggest predictor turns out to be the feature which wont be present when the data is created.
tomjakubowski 8y ago
There are some wonderful examples of this in tom7's videos about learnfun/playfun, an AI that (in short, very simple/stupid terms because that's all I know) in learnfun watches human gamepad inputs + the NES memory, and in playfun tries to provide its own inputs to make numbers in certain NES memory locations (those that went up for the human player) go up. One way to not lose is to just pause the game. Minor spoiler: learnfun figures that one out :-) http://www.cs.cmu.edu/~tom7/mario/ http://www.cs.cmu.edu/~tom7/mario/
adampk 8y ago
So "Specification Gaming" seems to be a new age The Monkey's Paw? https://en.wikipedia.org/wiki/The_Monkey%27s_Paw https://en.wikipedia.org/wiki/The_Monkey%27s_Paw
fatnoah 8y ago
This is my favorite: "A robot arm with a purposely disabled gripper found a way to hit the box in a way that would force the gripper open"

neel8986 8y ago

Great examples. I think algorithms in sites like Facebook did something similar. In order to maximize view/clicks, they themselves created problems like an echo chamber or promoting divisive articles which in turn made the whole experience worse

reitanqild 8y ago
And Google has "simplified" search by fuzzing terms until it is somewhat possible to use even if you really cannat spel. In the process they broke it so badly I now prefer DDG. (Who also gives me results I didn't ask for.)
sizzle 8y ago
Facebook algorithms making the whole experience worse is being quite charitable in describing what it did, considering it's huge role in spreading of fake news and biasing people's thinking towards the presidential election along with creating strong negative feedback loops.

renjimen 8y ago

Not quite the same, but equally as disastrous is data leakage. This comes in some fun and unexpected forms. One example is from the fisheries monitoring competition on Kaggle, where NNs were learning to predict fish caught using the background image of the boats rather than the fish shown on camera. i.e. different boats tend to catch different fish

devoply 8y ago
inductive learning.
thisisit 8y ago
I have recently started learning ML and trying competitions on Kaggle. And I also seeing that in many of the competitions I have tried. The biggest predictor turns out to be the feature which wont be present when the data is created.

tomjakubowski 8y ago

There are some wonderful examples of this in tom7's videos about learnfun/playfun, an AI that (in short, very simple/stupid terms because that's all I know) in learnfun watches human gamepad inputs + the NES memory, and in playfun tries to provide its own inputs to make numbers in certain NES memory locations (those that went up for the human player) go up. One way to not lose is to just pause the game. Minor spoiler: learnfun figures that one out :-) http://www.cs.cmu.edu/~tom7/mario/ http://www.cs.cmu.edu/~tom7/mario/

adampk 8y ago

So "Specification Gaming" seems to be a new age The Monkey's Paw? https://en.wikipedia.org/wiki/The_Monkey%27s_Paw https://en.wikipedia.org/wiki/The_Monkey%27s_Paw

fatnoah 8y ago

This is my favorite: "A robot arm with a purposely disabled gripper found a way to hit the box in a way that would force the gripper open"