4 ms·

A nice allegory about strong AI.

by philipswood 3y ago

yawpitch 3y ago
If only chess were a nice allegory for life.
- cranium 3y ago
  I remember a quote "Life is like Tetris, stop playing it like chess." I quite like the analogy: you don't have perfect information and what matters is how you deal with what you get. (After googling, it turned out to be a blog post: https://observer.com/2016/01/your-life-is-tetris-stop-playing-it-like-chess/ https://observer.com/2016/01/your-life-is-tetris-stop-playin...)
- mitthrowaway2 3y ago
  Given the context, what do you think are the relevant differences that affect the validity of the metaphor?
  - yawpitch 3y ago
    For one, the child can play any number of other games or choose not to play at all; the child can break the board, flip it upside down, fold it in half, introduce the concept of wraparound or flip the way pieces move… the child is free to choose to change the definition and rules of the game… the AI can only, ever, play chess, using only the rules it was given. The metaphor assumes that life is a game in which there are a finite number of moves and someone is playing to win… chess isn’t remotely a workable metaphor for life.
    - philipswood 3y ago
      I think his message is targeting a bit deeper than that. If I had so sum it up (always hard with an allegory): Developing superhuman AI is a game you can only play once and can expect to lose without ever having a good understanding of how, why or even when you lost. The suggestion is (like in the movie Wargames) that the best move is not to play. I'd say the child in the story is mankind. Developing strong AI is the game. It's a game we can expect to lose.
      - yawpitch 3y ago
        Interesting interpretation, but if true it’s poorly constructed: the child can’t be mankind, as the “you” in the game would then be god and god (not mankind) would have developed the (in the story already extant) AI… and AI that is not, obviously, a strong AGI. Now, it could be that “you” is Homo Not So Sapiens and the child is Homo Antesingularity… but even then I think you’re stretching it past its breaking points.
      - nvm0n2 3y ago
        > Developing superhuman AI is a game you can only play once But that's not right. Stockfish is superhumanly intelligent at playing chess but it is not the result of a single match between its creator and itself. Humans played chess against computers over and over, for decades, and the "superintelligence" level was seen coming a mile off. Moreover it was never dangerous and those AI techniques never led to a general intelligence. This is the problem with AI doomers like Yudkowsky. They use rhetorical skill to make clever sounding arguments that contain fallacies, and hope you won't notice. The assumption you can only "play once" is a wild one and thrown in at the end with absolutely nothing earlier in the story supporting it or backing it up.
- philipswood 3y ago
  Yeah, maybe poker would be a better choice of game.
- philipswood 3y ago
  I find that most fun games are "cartoons" of life. In any cartoon you choose which features to keep and emphasize and which features to leave out. Chess is a game of perfect potential knowledge - where the only limit to seeing is your own ability. What's nice about chess is the "if-I-do-this,-what-happens-in response" kind of thinking it instills.

A nice allegory about strong AI.

yawpitch 3y ago

If only chess were a nice allegory for life.

cranium 3y ago
I remember a quote "Life is like Tetris, stop playing it like chess." I quite like the analogy: you don't have perfect information and what matters is how you deal with what you get. (After googling, it turned out to be a blog post: https://observer.com/2016/01/your-life-is-tetris-stop-playing-it-like-chess/ https://observer.com/2016/01/your-life-is-tetris-stop-playin...)
mitthrowaway2 3y ago
Given the context, what do you think are the relevant differences that affect the validity of the metaphor?
- yawpitch 3y ago
  For one, the child can play any number of other games or choose not to play at all; the child can break the board, flip it upside down, fold it in half, introduce the concept of wraparound or flip the way pieces move… the child is free to choose to change the definition and rules of the game… the AI can only, ever, play chess, using only the rules it was given. The metaphor assumes that life is a game in which there are a finite number of moves and someone is playing to win… chess isn’t remotely a workable metaphor for life.
  - philipswood 3y ago
    I think his message is targeting a bit deeper than that. If I had so sum it up (always hard with an allegory): Developing superhuman AI is a game you can only play once and can expect to lose without ever having a good understanding of how, why or even when you lost. The suggestion is (like in the movie Wargames) that the best move is not to play. I'd say the child in the story is mankind. Developing strong AI is the game. It's a game we can expect to lose.
    - yawpitch 3y ago
      Interesting interpretation, but if true it’s poorly constructed: the child can’t be mankind, as the “you” in the game would then be god and god (not mankind) would have developed the (in the story already extant) AI… and AI that is not, obviously, a strong AGI. Now, it could be that “you” is Homo Not So Sapiens and the child is Homo Antesingularity… but even then I think you’re stretching it past its breaking points.
    - nvm0n2 3y ago
      > Developing superhuman AI is a game you can only play once But that's not right. Stockfish is superhumanly intelligent at playing chess but it is not the result of a single match between its creator and itself. Humans played chess against computers over and over, for decades, and the "superintelligence" level was seen coming a mile off. Moreover it was never dangerous and those AI techniques never led to a general intelligence. This is the problem with AI doomers like Yudkowsky. They use rhetorical skill to make clever sounding arguments that contain fallacies, and hope you won't notice. The assumption you can only "play once" is a wild one and thrown in at the end with absolutely nothing earlier in the story supporting it or backing it up.
philipswood 3y ago
Yeah, maybe poker would be a better choice of game.
philipswood 3y ago
I find that most fun games are "cartoons" of life. In any cartoon you choose which features to keep and emphasize and which features to leave out. Chess is a game of perfect potential knowledge - where the only limit to seeing is your own ability. What's nice about chess is the "if-I-do-this,-what-happens-in response" kind of thinking it instills.