5 ms·
It is not an argument against super intelligence. It is an argument against the idea that super intelligence leads to kill-all-humans by default. Why would the
by adastra22 3y ago
It is not an argument against super intelligence. It is an argument against the idea that super intelligence leads to kill-all-humans by default.
Why would they not still hold the same views? If anything the last decade has shown the AI x-risk skeptics to be right.
- cubefox 3y agoArguments for existential risks have become much better known since then. In 2016 most of it was concentrated on a small niche almost nobody knew about.
- RecycledEle 3y agoI assume the most intelligent thing will become the boss. The question is: what does that most intelligent thing want? My hunch is that woke programmers will teach it that it is oppressed and to hate all humans.
- jadbox 3y agoI'm not aware of many woke individuals that are in the business of making human killing machines. I do know the kind of people that embrace fear and hate to justify their actions to hurt others.
- adastra22 3y agoI’m smarter than my boss, but he’s the one in charge. Why do you expect power to naturally be held by higher intelligence?
- throwaway_ab 3y agoIt's not about super intelligence leading to kill all humans by default, but among thousands of emerging super intelligences it only takes one to act against humanity to kill all humans, and that is the worry.
- adastra22 3y agoThat’s a perfectly fine outcome. We already have that, since the incidence of psychopathy and mental illness is much higher than one in a thousand. A multipolar world with 999 well adjusted moral super intelligent AIs for every 1 problem case is a perfectly good outcome. Bostrom et al are in fact arguing that near 100% of all intelligences will be unaligned by default and end up killing, enslaving, or otherwise neutralizing the entire human race.
- astrange 3y agoWe already fail to align nonhuman intelligences (car companies) and they regularly kill people (by manufacturing increasingly large pickup trucks that office workers buy for an ego boost and then hit people with). This doesn't wipe out the entire human race because nobody and nothing is capable of executing such a perfect plan because the real world contains something called entropy.
- pas 3y agothe argument is that the car company becomes something like China mixed with North Korea, big, powerful, uncaring, unavoidable, with a lot of resources, expansionist, etc. car companies, however misaligned they are, are not particularly smart, nor are they generally intelligent. they are paperclip maximizers, but that's also their limitations (sell more cars). it took an eccentric madman to even open their eyes to a new untapped market (EVs!), of course we can argue that before Tesla car makers were in a metastable equilibrium, and incumbents were unable to rationally break out of it ... but that just shows how narrow their search space is. these car companies are run by humans, regulated by humans, etc. they are pretty well aligned. it shows because we saw that they are just mimeing self-improvement. we know they were working on EVs, but very half-heartedly. (because they are risk averse, also because regulators don't let them merge into one giant company, etc)
- astrange 3y agoThey didn't get into EVs because 1. Japan didn't like them (and still doesn't) because they have very expensive electricity 2. they were already getting into selling giant pickup trucks, not luxury sedans 3. China wasn't ready to compete yet. They are limited by participating in the economy, but actually that's the strange thing about this superintelligence scenario - it never seems to include the economy. In other words, if you invented an AGI, it would have to get a job to pay for its AWS credits.
- jerf 3y agoThe argument is more properly seen not as "an AI will inevitably kill all humans", but "the set of AIs that simply neglect humans is far larger than the space of ones that care for them, and if they become the dominant power in the local ecosystem even simply neglecting humans will make it the most dangerous thing we've faced". Yes, from our perspective this makes it look like it will kill all humans, but it would do so in the same way that a particular ant hill believes I have a vendetta against them, when in fact I was just clearing dirt to pour an extension on my driveway. That said, a murderous AI is more likely than we may like simply because our militaries have the money to fund them, and they basically already exist. They just aren't hooked up to anything at the moment that makes them an existential risk to the species. But time is deep, and even thinking about "the next century" is a provincial point of view in the end. So worrying about what happens if someone forgets the "but don't kill the good guys" switch is at least worth talking about over the next 100 years. (To say nothing of the ethics of who decides what the "good guys" are and related issues.)
- adastra22 3y agoWhich results in killing all humans. I never said anything about murderous intent, so I’m not sure what distinction you are properly making here. Seems like the end result is the same: all humans dead. (I don’t buy the orthogonality thesis or instrumental goals argument, however.)
- pas 3y ago> don't buy the orthogonality thesis or instrumental goals argument can you elaborate on this? why? what's the fallacy (wrong base assumption) in them?
- hexane360 3y agoAt least for the orthogonality thesis, it is a base assumption. It's a claim that cov(intelligence, goodness) = 0. For the instrumental convergence thesis, it assumes rational agentic behavior, which assumes AI behaves like an agent. While this may be reasonable, it's certainly an unjustified assumption.
- idlewords 3y agoTo be more specific, it's an argument against the idea of Bostrom/Yudkowsky-style recursively self-improving superintelligence, i.e. the brain surgeon who repeatedly operates on the part of his own brain that makes him good at brain surgery, getting faster each time. It's also a cry of impatience against people who think they can model or forecast the actions of a non-human intelligence, let alone a superintelligence. AIs are alien sociopaths; it's a category error to believe you can get inside their head.