3 ms·
But you're now presupposing the AI finds humanity curious. Which is not a given for a random mind that came from outer space and visited our solar system. As f
by TeMPOraL 6y ago
But you're now presupposing the AI finds humanity curious. Which is not a given for a random mind that came from outer space and visited our solar system.
As for superhuman AIs developed here, on Earth, by us, it's very improbable that we could go from no AI to AI that can manage mass and energy at will (and is curious about us). The reasonable scenario is gradual (if exponentially accelerating) development of such AI. Which means it won't initially have unlimited access to the universe - it'll have to grow using resources on hand. Which is us, and our infrastructure.
In other words, all atoms may be alike, but those that make you are the easiest to get.
- beefield 6y ago> But you're now presupposing the AI finds humanity curious. Yes. I am assuming that a vastly more intelligent thing than human would be almost by definition more, not less, interested in pretty much everything. Obviously that thing has more capacity to be interested and understand different things. Super intelligent thing not interested in how different things work just... does not sound that intelligent to me.
- TeMPOraL 6y agoI currently believe the "orthogonality thesis"[0] of Nick Bostrom, which argues that in an arbitrary mind, goals and values are independent of the capacity to achieve them. Curiosity as we experience it not really instrumental (even if it evolved that way), and more of a value - that is, we are curious about something "in general", not because we need it for something. I can easily imagine a superhuman paperclip maximizer AI, which by definition would have no general curiosity about humans, to look as if it was curious about us - for as long as it was captive, and just in order to learn how to manipulate us better. That curiosity would end the moment it seized the means of paperclip production. In almost every imaginable case, you really don't want an AI to be instrumentally curious about us. Best if it didn't notice us at all, and left to seek its business elsewhere. -- [0] - https://www.lesswrong.com/tag/orthogonality-thesis https://www.lesswrong.com/tag/orthogonality-thesis