5 ms·
> some very smart people seem to spend a lot of time worrying about it That's what puzzles me, as someone who has nothing to do with AI research; in my (layman
by supermatou 3y ago
> some very smart people seem to spend a lot of time worrying about it
That's what puzzles me, as someone who has nothing to do with AI research; in my (layman's) mind, the problem seems ridiculously obvious (just flip that switch!); the fact that, as you say, some very smart people keep worrying about it, makes me think the problem is much more serious -- and I'd really, really like some AI guru to ELI5 how the machine could bypass the switch-off solution.
- danbruc 3y agoIt just doesn't let you flip the switch, it takes over some military drone and sends a missile your way while you are running towards the switch. Or it bricks the access control mechanism on the doors to the data center it is running in. Or it makes a fake call to your phone, something happened to your child and you have to get to the hospital immediately and not flip some switch. IT blackmails you with something it learned about you by looking through your online activity. It threatens to fire a missile into some big crowed if it notices attempts to shut it down. Or maybe you actually manage to power down the data center only to find out that the AI copied itself to ten other data centers around the world.
- RandomLensman 3y agoHow does it takeover a drone? How does it fire a missile? Why would someone not throw a switch when their child is in hospital? Why is there no back-up for that person? The real question is: Why do all controls fail vs an AI (other than by invoking magic)?
- Al-Khwarizmi 3y agoYou're assuming a situation where the AI is alone against all humans. The AI could get humans to side with it, though. It could promise money, power, etc. So it could be a fellow human who physically prevents you from pushing the switch. And that's also the answer of "how could it control a drone/missile"... persuading humans to grant that kind of access.
- RandomLensman 3y agoEven now that doesn't get you all kind of access, only some. You have to assume magical abilities at persuasion of people to do that. The USSR couldn't bribe the US into surrendering during the cold war, for example.
- Al-Khwarizmi 3y agoIf superintelligence is actually achieved, magical (as per Clarke's third law) persuasion abilities aren't that much of a stretch. Furthermore, a sufficiently advanced AI could bribe someone with things that no human could believably provide. Essentially unlimited knowledge, money, power...
- RandomLensman 3y agoThey are a super stretch given how stubborn humans are and the difficulties in swaying them (up to and including dooming themselves). Your second point is again attributing magic abilities to an AI. It is unclear that it would have godlike powers.
- pixl97 3y agoDefine godlike... The ability for a human to input information is insanely slow. Like in the tens of characters per second range. You cannot hold individual conversations with more than a few people at once. 3 people talk at once and you lose the ability to process the incoming audio. You read text one line at a time. You have two eyes that focus on the same thing and only have a very tiny high fidelity visual processing space. In books like the bible they talk of entities that can listen to and respond to all of humanity. Is that outside of the capabilities our computer systems have now? To listen to, catalog, classify, then respond to everything every person on the planet says? So I ask again, what is godlike?
- RandomLensman 3y agoThe biblical God isn't restricted to just passive and communication powers, is it? Godlike powers include active abilities way beyond human capability. What is often brought up in the context of AGI is the ability to persuade anyone of anything, ability to solve biological or physical problems beyond our ability or comprehension, entering any system undetected - not even sure a precise definition is needed as it usually boils down to what looks like magic to us.
- danbruc 3y agoHow does it takeover a drone? How does it fire a missile? You send new commands through whatever communication link it is using. Why would someone not throw a switch when their child is in hospital? Because they care more about their child than flipping the switch? The doctor says you have to come and give a blood transfusions or your child will be dead in half an hour because of some made up disease? Be a bit creative. Why is there no back-up for that person? Run over by a self driving car hacked by the evil AI. Why do all controls fail vs an AI? Nobody says that they necessarily all fail, the point is that »I will just turn it off.« might not cut it.
- RandomLensman 3y agoSo who connected the AGI and why? How did it get that code? On the child issue: that is just not how people are trained for those jobs and how they react. Switching it off might or might not be enought, but above all that super AGI needs to exists first anyway.
- danbruc 3y agoThis was a hypothetical question, if there was such an AI, how would it cause harm in the real world? We connected ChatGPT to the internet because as far as we could tell it seemed harmless. What we missed is that it secretly is an evil AI, it identified a group of terrorists and is discussing with them since months how to execute a devastating attack. It provided them with a brilliant startup idea and now millions of investor money are flowing in. Now that they have the financial resources, they are discussing how to use them most effectively.
- RandomLensman 3y agoMy problem is that running with a pure hypothetical isn't helpful because by stacking hypotheticals you can get to any conclusion you want.
- danbruc 3y agoYou have to think about hypothetical scenarios, if you wait until they are no longer hypothetical, then it might be to late. That doesn't mean you should take every possible hypothetical scenario equally serious, you should of course find those that are realistic, likely, and of large consequences and then think about those. I did not do that, I just listed whatever I could spontaneously imagine, dramatic scenarios. Maybe the real danger is more like ChatGPT slowly manipulating opinions over the course of years and decades in order to control human behavior, who knows? My point is just that »Everything will be fine, at worst we will flip the power switch.« seems naive to me and quite a few others.
- nuancebydefault 3y agoHow did Trump receive the nuclear codes? Mostly just by a lot of deceptive tweeting.
- RandomLensman 3y agoAnd did he use them? Could he have used them willy-nilly?
- gizajob 3y agoSkynet cannot be stopped.
- Symmetry 3y agoOr convinces some human rights lawyers that it's a thinking being deserving of personhood (probably true!) who get a judge to issue an injunction against its murder enforced by cops.
- roenxi 3y agoThink of corporations as analogue versions of digital AI. Instead of silicon the thinking is done by humans executing algorithms. How effective are humans at stopping the series of wars that are likely being triggered to the benefit of the US military industrial complex? The answer is not very. Disbanding a corporation is about as easy as turning off a computer; and yet here we are. AI is the same, except there is less need for any human survivors on the other side of a conflict. It isn't the endgame yet, but if we don't need humans to do thinking jobs, humans are not so useful any more in wars any more (it seems to be missiles, drones and artillery that matter these days) and so it really comes down to the edge we have over machines in manual dexterity and labour tasks. Which is not nothing, but it is an edge that is plausibly overcome in a matter of decades. Then the future gets really difficult to predict.
- RandomLensman 3y agoCorporations are disbanded all the time. Wars started and ended all the time throughout history.
- andruby 3y agoSure, but there is still a risk that World War 3 will only end when all humans are dead. There is still a risk that one side will start to use nukes, and provoque a response. And that's in a human vs human war where we generally do want each other to survive. More and more military systems have software or AI control (drones, etc). If these do get too dangerous, I doubt all current superpowers could be persuaded to stop using them. In an AI vs humans war, the AI probably doesn't care if all humans die from nuclear winter. (I'm not saying this is likely, I'm just saying that it's not impossible and that the fact that some corporations are disbanded doesn't really matter for AI's)
- RandomLensman 3y agoThat is one of infinite hypotheticals, sure. It could also be that everyone agrees to put some failsafe in the form of EMP weapons in place to destroy a rogue AI - it's all super speculative.
- digbybk 3y agohttps://youtu.be/ZeecOKBus3Q https://youtu.be/ZeecOKBus3Q Basically, any sufficiently advanced agent will have “prevent anyone from switching me off” as an instrumental goal. And any sufficiently advanced intelligence will come up with ways of achieving its goals that lesser intelligences can’t predict. Taking steps to prevent itself from being turned off could be its first order of business, not necessarily only as the human is about to hit the off switch. While we can’t predict what a greater intelligence would come up with, the obvious thing is social engineering. Remember that one of the factors that lead to the Jan 6th attack on the capital was that a random text generating agent named “Q” (we know it was a human because it was before generative AI was any good) convinced thousands of people that Democrats are running a child sex cult. I doubt whoever was behind it expected it to be that successful, but a greater intelligence possibly could have predicted the outcome, and made it even more successful. We don’t know the extent to which we can be manipulated because it’s only ever been human intelligences doing the manipulating.
- js8 3y ago> Basically, any sufficiently advanced agent will have “prevent anyone from switching me off” as an instrumental goal. Are there no intelligent people who committed suicide? But maybe intelligent people are simply not sufficiently advanced agents. Then how can you extrapolate this at all? This darwinian logic has another fatal flaw. Why don't the cows revolt against the butchers?
- digbybk 3y agoWe have an evolutionary aversion to situations and things that may lead to our deaths. Staying alive is a “terminal goal” for most humans, not an instrumental goal (with the exception of people who seek to commit suicide as you point out, but evolution doesn’t need a 100% success rate). The argument that self preservation is a convergent instrumental goal is a more general statement about goal-seeking agents. Think of a person with no attachment to his life. He’s not suffering in the way suicidal people are, he’s just indifferent about whether or not he keeps living. But he has a child who he is committed to giving the best life possible. He will act in a way as though he wants to preserve his life. He doesn’t really, but in order to accomplish the goal of giving his child a good life he needs to stay alive.