3 ms·

I don't think the analogy to AI is apt because Stockfish isn't intelligent and life isn't a chess game. A one million IQ AI might be able to hack your chess gam

I don't think the analogy to AI is apt because Stockfish isn't intelligent and life isn't a chess game. A one million IQ AI might be able to hack your chess game (or your OS) and win that way, or persuade you by chat to let it win (it could give you the cure for cancer and help you make some investments in the stock market - isn't that worth more than winning a game?) Life isn't a chess games either. It's not clear that a contest is happening or that only side will win. Humanity could, now, quickly stop AI. We could burn down all the data centers and chip fabs and kill everyone who knows linear algebra. But we don't because nobody ever sits down and says "Okay, the contest for existence starts now!" Instead, AI capabilities increase year after year and month after month and the AI will simply be able to bide its time to start the game. To use the chess metaphor the table is set where humanity has all the pieces and AI only its king - but every hour AI adds another piece. If we never make a move we will face it at full strength.

upwardbound 3y ago
> AI capabilities increase year after year and month after month and the AI will simply be able to bide its time to start the game. To use the chess metaphor the table is set where humanity has all the pieces and AI only its king - but every hour AI adds another piece. If we never make a move we will face it at full strength. This is an excellent analogy and you’re right that it’s imperative we nip this in the bud.
- zmgsabst 3y ago
  This logic says you should beat your child in case they grow up to be a violent criminal: A self-fulfilling prophecy that precipitates the very conflict it ostensibly avoids.
  - the8472 3y ago
    Omega appears and tells you your child will have an probability P to be the next Mao, only conditional on the child surviving to adulthood. At which P do you drown the child?
  - ALittleLight 3y ago
    Not at all. My children aren't on the path to becoming more capable than humanity. If my children are violent criminals I expect human civilization will be able to handle them. I don't have that expectation for AI.
pixl97 3y ago
This is one of the arguments I've used in the past. It's easy to kill ASI if it's a data center. But if it's running on a cellphone?
freilanzer 3y ago
> Instead, AI capabilities increase year after year and month after month and the AI will simply be able to bide its time to start the game. To use the chess metaphor the table is set where humanity has all the pieces and AI only its king - but every hour AI adds another piece. If we never make a move we will face it at full strength. This is a very optimistic view. I'm not so sure that true AGI will be developed even in this century. GPT-4 and the likes are hitting a data limit, and are not close to being AGI.

upwardbound 3y ago

> AI capabilities increase year after year and month after month and the AI will simply be able to bide its time to start the game. To use the chess metaphor the table is set where humanity has all the pieces and AI only its king - but every hour AI adds another piece. If we never make a move we will face it at full strength. This is an excellent analogy and you’re right that it’s imperative we nip this in the bud.

zmgsabst 3y ago
This logic says you should beat your child in case they grow up to be a violent criminal: A self-fulfilling prophecy that precipitates the very conflict it ostensibly avoids.
- the8472 3y ago
  Omega appears and tells you your child will have an probability P to be the next Mao, only conditional on the child surviving to adulthood. At which P do you drown the child?
- ALittleLight 3y ago
  Not at all. My children aren't on the path to becoming more capable than humanity. If my children are violent criminals I expect human civilization will be able to handle them. I don't have that expectation for AI.

pixl97 3y ago

This is one of the arguments I've used in the past. It's easy to kill ASI if it's a data center. But if it's running on a cellphone?

freilanzer 3y ago

> Instead, AI capabilities increase year after year and month after month and the AI will simply be able to bide its time to start the game. To use the chess metaphor the table is set where humanity has all the pieces and AI only its king - but every hour AI adds another piece. If we never make a move we will face it at full strength. This is a very optimistic view. I'm not so sure that true AGI will be developed even in this century. GPT-4 and the likes are hitting a data limit, and are not close to being AGI.