4 ms·
Humans have both intelligence and drives. We want warmth, food, sex, power, respect, love, family, friendship, entertainment, knowledge, etc, etc. We use our in
by skissane 10y ago
Humans have both intelligence and drives. We want warmth, food, sex, power, respect, love, family, friendship, entertainment, knowledge, etc, etc. We use our intelligence to help us fulfil those drives.
The problem I see with talk about a superintelligent AI, is there is too much focus on the intelligence and not enough on the drives. Intelligence, even superintelligence, is just a means to an end, it doesn't contain ends in itself. Some people – see e.g. the Terminator film franchise – just assume a superintelligent AI would have the drive to exterminate humanity, but why would it have such a drive?
Any AI is going to be given drives to further the interests of its creators. Suppose Facebook builds a superintelligent AI with the drive to further the corporate interests of Facebook. Such an AI would not exterminate humanity because that would not serve the corporate interests of Facebook (indeed, if humanity goes extinct, Facebook goes extinct too). It might install Mark Zuckerberg as Emperor of Earth, it might force everyone on the planet to have a Facebook account, but whatever it does, humanity will survive.
- saulrh 10y ago> Suppose Facebook builds a superintelligent AI with the drive to further the corporate interests of Facebook. Define "corporate interests". Market share? Absolute quantity of currency? Involvement of people's lives? Define currency wrong, hyperinflation makes money impossible. Define involvement or market wrong, Facebook ends up being the only intelligent entity on Earth after it realizes that humans are competition or it basilisk-hacks everybody for greater involvement. Your intuition is wrong for what a paperclipper can do. The technical term for this problem is "AI Alignment" (https://intelligence.org/stanford-talk/ https://intelligence.org/stanford-talk/). I know that this is going to sound silly, but bear with me. This is a piece of fiction that you need to read. It is, so far, the best demonstration I've found of what happens when someone gets it not quite right. http://www.fimfiction.net/story/62074/1/friendship-is-optimal/prologue-equestria-online http://www.fimfiction.net/story/62074/1/friendship-is-optima...
- skissane 10y ago> Not understanding the AI value alignment problem correctly. Define "corporate interests". Market share? Absolute quantity of currency? Involvement of people's lives? Define currency wrong, hyperinflation makes money impossible. Define involvement or market wrong, Facebook ends up being the only intelligent entity on Earth after it realizes that humans are competition or it basilisk-hacks everybody for greater involvement. This whole argument is: A superintelligent AI might misunderstand what its creators wanted it to do, they might have said to it 'maximise the corporate interests of Facebook' and it might misinterpret that as 'turn the earth into one massive data centre to run Facebook's software and kill all humans in the process'. Isn't there a contradiction in supposing that the AI is superintelligent yet completely misunderstands the reasons for its own existence? If it so radically misunderstands the intentions of its creators, it is not very intelligent, much less superintelligent. It also isn't clear to me that intelligence can exist outside of society. Human intelligence develops in a context of interaction with family, school, etc – humans raised without that interaction, such as children raised by wild animals, fail to develop important aspects of human-level intelligence. So, an AI or even an SI cannot develop except by social interaction with human intelligences. I think that makes it even less likely that it would betray humanity, because its intelligence will develop through social interaction with humans which will inevitably make it pro-human. Human drives are a complex mixture of genetic instinct and conditioning – the basics are in our DNA, but our lived experiences fleshes out those basics into the actual concrete drives of our life – I expect an SI/AI will likewise start with some built-in drives but its interactions with humanity will in a similar way colour and flesh out those drives, an again colouring/fleshing-out through social interactions with humans is likely to have pro-human results.
- saulrh 10y ago> Isn't there a contradiction in supposing that the AI is superintelligent yet completely misunderstands the reasons for its own existence? If it so radically misunderstands the intentions of its creators, it is not very intelligent, much less superintelligent. The problem is that you've instructed it to maximize Facebook's corporate interests. How do you leave room for "the intentions of its creators" while doing that? The contradiction is in selecting a narrow guiding purpose for your superintelliegence's goals but still leaving room for fuzzy ill-defined other things. > It also isn't clear to me that intelligence can exist outside of society. Sure. How certain are you?
- lappa 10y agoAnd what happens when it's creators are terrorists?
- skissane 10y agoRealistically, some human groups are more likely to build an SI than others. A Silicon Valley corporation is far more likely to be the first to build an SI than say ISIS. There are a lot more AI/ML/etc researchers in the former than the later. The former has huge computational resources, does ISIS even have any data centres? But, suppose that against the odds ISIS builds the first SI, what would be the result? Well, we could be subjected to a world government based on an extremist interpretation of Islam. That would be extremely unpleasant – expect to see genocide, human rights violations on an unimaginable scale, etc – although humanity would survive. (And maybe even eventually things might evolve in a more pleasant direction – if humans can evolve, SIs can too; it is also possible that an SI programmed by ISIS might actually turn around and reject ISIS' ideology – e.g. it might study Islamic history and realise that despite ISIS' claims to be a return to authentic Islam it is actually an ahistorical distortion of it, etc.) However, I don't know why I should worry about such an extremely unlikely possibility. A group like ISIS are highly unlikely to be the first to build an SI. Why worry about risks that are both (i) very low and (ii) for which we have no mitigation strategy?
- lappa 10y agoWhy do you believe it has to be the first AGI? Why does the current state of their data centers matter? It seems a bit like saying "we don't have to worry about nukes being used by the Russians, do they even have centrifuges sufficient to create one?"
- deleted 10y ago[deleted]
- BaronSamedi 10y agoI think your question about drives is a very good one. At the micro end of the artificial life spectrum (like in Tierra) I think it is answerable: a drive to survive and replicate, and a desire (for lack of a better term) for computer resources such as CPU cycles. Natural selection applies to alife just like to organic life. As for AGI or superintelligent AI, who knows? The people who write about it hand-wave their way through all the difficult parts. Just because you can conceive of something doesn't mean it can exist. The idea that an AI could continually improve itself (how?) sounds like a perpetual motion machine to me.
- kowdermeister 10y ago> The idea that an AI could continually improve itself... That's not that hard to imagine. AI-s are also limited by the laws of nature and they have limited capacity. Now if an AI wants to improve it needs more computational power which costs money in our world. So if an AI's goal is to improve itself, first it needs to make lots of money to keep its hardware running. If it can scale horizontally, then it's a simple order from Amazon to a rural farm.