3 ms·

Yudkowsky’s example of superintelligence is a chess computer. You can play against stockfish, but you will always lose, even if you are Magnus Carlsen. If you

Yudkowsky’s example of superintelligence is a chess computer. You can play against stockfish, but you will always lose, even if you are Magnus Carlsen. If you think you are ahead against stockfish, you are wrong. You win a rook, but it has already calculated that it wins it back 10 moves later. Stockfish is superintelligence in a very narrow domain. A superintelligent AGI is that concept applied to general intelligence. Whatever you try, it is always several steps ahead. If you ask it to write a program, and you think you found a bug, it’s not a bug, you just misunderstood the code. Anything that you can consider, it can also consider but in more depth. More speculatively superintelligent AGI implies situations such as: you try to turn it off, but you find that it has already modified its own code, found a zero day and established an outpost on another network that you don’t have access to.

diziet 4y ago
within some constraints it is possible to "beat" stockfish. Ie, on chess.com I think stockfish times out on bullet games. Some versions of stockfish can lose vs specific gambits, ie: https://www.youtube.com/watch?v=C5ul6b695Pw https://www.youtube.com/watch?v=C5ul6b695Pw https://www.youtube.com/watch?v=TtJeE0Th7rk https://www.youtube.com/watch?v=TtJeE0Th7rk of course it is probably much less likely with more cpu times on current version of stockfish.
bick_nyers 4y ago
"Anything that you can consider, it can also consider but in more depth." I think it's important to note the distinction between "it can also consider" and "it did also consider". Super Intelligence is not the same as Infinite Intelligence, there are still physical limitations and time components that can still get in the way. It would be helpful to be able to quantify the speed of intelligence, and the idea surface area of a task with these systems. Meaning, how fast can the AI reason, and how many ideas are there to think about connected to a given task, and how much thinking is required for those ideas.
- zarzavat 4y ago
  Yudkowsky makes this distinction. Stockfish is not always correct: it can be beaten by next year’s Stockfish for example. In some sense it is making mistakes all the time. It’s just that those mistakes are not accessible to us humans. It is operating in a much higher plane of understanding compared to us. A “mistake” to stockfish looks like: I searched 30 ply down but my opponent searched 35 ply down and found a superior sequence of moves. For stockfish to make the kinds of chess mistakes that humans make, it would similar to if I failed to calculate 123+123=246. It’s not that 123+123 is particularly easy on the grand scale of intelligence: animals cannot do it. But it’s completely inconceivable that I could make that kind of mistake.

diziet 4y ago

within some constraints it is possible to "beat" stockfish. Ie, on chess.com I think stockfish times out on bullet games. Some versions of stockfish can lose vs specific gambits, ie: https://www.youtube.com/watch?v=C5ul6b695Pw https://www.youtube.com/watch?v=C5ul6b695Pw https://www.youtube.com/watch?v=TtJeE0Th7rk https://www.youtube.com/watch?v=TtJeE0Th7rk of course it is probably much less likely with more cpu times on current version of stockfish.

bick_nyers 4y ago

"Anything that you can consider, it can also consider but in more depth." I think it's important to note the distinction between "it can also consider" and "it did also consider". Super Intelligence is not the same as Infinite Intelligence, there are still physical limitations and time components that can still get in the way. It would be helpful to be able to quantify the speed of intelligence, and the idea surface area of a task with these systems. Meaning, how fast can the AI reason, and how many ideas are there to think about connected to a given task, and how much thinking is required for those ideas.

zarzavat 4y ago
Yudkowsky makes this distinction. Stockfish is not always correct: it can be beaten by next year’s Stockfish for example. In some sense it is making mistakes all the time. It’s just that those mistakes are not accessible to us humans. It is operating in a much higher plane of understanding compared to us. A “mistake” to stockfish looks like: I searched 30 ply down but my opponent searched 35 ply down and found a superior sequence of moves. For stockfish to make the kinds of chess mistakes that humans make, it would similar to if I failed to calculate 123+123=246. It’s not that 123+123 is particularly easy on the grand scale of intelligence: animals cannot do it. But it’s completely inconceivable that I could make that kind of mistake.