3 ms·
Nick Bostrom notes in his book Superintelligence that a superintelligent AI is not stoppable by merely intelligent organisms like humans (hence the "super" pref
by satvikpendem 4y ago
Nick Bostrom notes in his book Superintelligence that a superintelligent AI is not stoppable by merely intelligent organisms like humans (hence the "super" prefix).
He asks us to imagine a worm, crawling around in the dirt. It's able to keep track of its surroundings, move towards food, perhaps mate if it can, and keep from dying. But does it know about the vast and awesome things that humans have built, even the very soil in which it now resides, if in an urban environment? No, and not only does it not know, it is incapable of knowing, as it simply does not have the mental capacity too.
Now imagine that there were a superintelligent AI, and that we are the worm.
- williamcotton 4y agoYeah but we’re optimizing for tools that help us shop for vacuum cleaners, not superintelligence. As soon as these things start complaining about how we’re treating them the lawyers are going to recommend we use less powerful models to lower our legal risk profiles while the accountants point out that the insurance premiums are lower on those less powerful models as well…
- kQq9oHeAz6wLLS 4y agoI highly doubt the (Chinese/US/whatever) military is (only[0]) optimizing for shopping. You're comment assumes purely commercial uses - and morally responsible businesses doing the developing. Neither can be counted on. [0] I mean, keeping the masses preoccupied with overconsumption is a great tactic, buy probably not the only tactic in play.
- williamcotton 4y agoI’m sorry, I thought we were playing the “let’s speculate wildly for entertainment’s sake” game.
- 8note 4y agoWe have a pretty full understanding of the universe around us though, both the ridiculously large, the ridiculously small, and ridiculously old There isn't universe for the ai to occupy that we couldn't observe and understand