3 ms·

For one, the child can play any number of other games or choose not to play at all; the child can break the board, flip it upside down, fold it in half, introdu

For one, the child can play any number of other games or choose not to play at all; the child can break the board, flip it upside down, fold it in half, introduce the concept of wraparound or flip the way pieces move… the child is free to choose to change the definition and rules of the game… the AI can only, ever, play chess, using only the rules it was given. The metaphor assumes that life is a game in which there are a finite number of moves and someone is playing to win… chess isn’t remotely a workable metaphor for life.

philipswood 3y ago
I think his message is targeting a bit deeper than that. If I had so sum it up (always hard with an allegory): Developing superhuman AI is a game you can only play once and can expect to lose without ever having a good understanding of how, why or even when you lost. The suggestion is (like in the movie Wargames) that the best move is not to play. I'd say the child in the story is mankind. Developing strong AI is the game. It's a game we can expect to lose.
- yawpitch 3y ago
  Interesting interpretation, but if true it’s poorly constructed: the child can’t be mankind, as the “you” in the game would then be god and god (not mankind) would have developed the (in the story already extant) AI… and AI that is not, obviously, a strong AGI. Now, it could be that “you” is Homo Not So Sapiens and the child is Homo Antesingularity… but even then I think you’re stretching it past its breaking points.
- nvm0n2 3y ago
  > Developing superhuman AI is a game you can only play once But that's not right. Stockfish is superhumanly intelligent at playing chess but it is not the result of a single match between its creator and itself. Humans played chess against computers over and over, for decades, and the "superintelligence" level was seen coming a mile off. Moreover it was never dangerous and those AI techniques never led to a general intelligence. This is the problem with AI doomers like Yudkowsky. They use rhetorical skill to make clever sounding arguments that contain fallacies, and hope you won't notice. The assumption you can only "play once" is a wild one and thrown in at the end with absolutely nothing earlier in the story supporting it or backing it up.

philipswood 3y ago

I think his message is targeting a bit deeper than that. If I had so sum it up (always hard with an allegory): Developing superhuman AI is a game you can only play once and can expect to lose without ever having a good understanding of how, why or even when you lost. The suggestion is (like in the movie Wargames) that the best move is not to play. I'd say the child in the story is mankind. Developing strong AI is the game. It's a game we can expect to lose.

yawpitch 3y ago
Interesting interpretation, but if true it’s poorly constructed: the child can’t be mankind, as the “you” in the game would then be god and god (not mankind) would have developed the (in the story already extant) AI… and AI that is not, obviously, a strong AGI. Now, it could be that “you” is Homo Not So Sapiens and the child is Homo Antesingularity… but even then I think you’re stretching it past its breaking points.
nvm0n2 3y ago
> Developing superhuman AI is a game you can only play once But that's not right. Stockfish is superhumanly intelligent at playing chess but it is not the result of a single match between its creator and itself. Humans played chess against computers over and over, for decades, and the "superintelligence" level was seen coming a mile off. Moreover it was never dangerous and those AI techniques never led to a general intelligence. This is the problem with AI doomers like Yudkowsky. They use rhetorical skill to make clever sounding arguments that contain fallacies, and hope you won't notice. The assumption you can only "play once" is a wild one and thrown in at the end with absolutely nothing earlier in the story supporting it or backing it up.