4 ms·

I think his message is targeting a bit deeper than that. If I had so sum it up (always hard with an allegory): Developing superhuman AI is a game you can only

I think his message is targeting a bit deeper than that. If I had so sum it up (always hard with an allegory): Developing superhuman AI is a game you can only play once and can expect to lose without ever having a good understanding of how, why or even when you lost. The suggestion is (like in the movie Wargames) that the best move is not to play. I'd say the child in the story is mankind. Developing strong AI is the game. It's a game we can expect to lose.

yawpitch 3y ago
Interesting interpretation, but if true it’s poorly constructed: the child can’t be mankind, as the “you” in the game would then be god and god (not mankind) would have developed the (in the story already extant) AI… and AI that is not, obviously, a strong AGI. Now, it could be that “you” is Homo Not So Sapiens and the child is Homo Antesingularity… but even then I think you’re stretching it past its breaking points.
nvm0n2 3y ago
> Developing superhuman AI is a game you can only play once But that's not right. Stockfish is superhumanly intelligent at playing chess but it is not the result of a single match between its creator and itself. Humans played chess against computers over and over, for decades, and the "superintelligence" level was seen coming a mile off. Moreover it was never dangerous and those AI techniques never led to a general intelligence. This is the problem with AI doomers like Yudkowsky. They use rhetorical skill to make clever sounding arguments that contain fallacies, and hope you won't notice. The assumption you can only "play once" is a wild one and thrown in at the end with absolutely nothing earlier in the story supporting it or backing it up.