2 ms·
I agree that it's not impossible for us to understand something smarter than us. But I don't particularly like your examples. Understanding a static well-organi
by csallen 12y ago
I agree that it's not impossible for us to understand something smarter than us. But I don't particularly like your examples. Understanding a static well-organized human-designed system like a 787 or a chess-playing algorithm is far simpler than understanding the thoughts and desires of a dynamic intelligence.
A better analogy would be to stick to IQ. How long would it take a group of people with an average IQ of 70 to understand and predict the workings of the mind of a genius with an IQ of 210? Probably a very long time, if ever. What if the genius was involved in direct competition with the group of people? She'd be incentivized to obscure her intentions and run circles around the group, and she'd likely succeed at both.
Just how intelligent might an AGI become given enough compute power? A 3x IQ increase is a conservative prediction. 10x, 100x, or even 1000x aren't unimaginable. How can pretend to know what such an intelligence might think about or care about?