81 ms·

How does it takeover a drone? How does it fire a missile? Why would someone not throw a switch when their child is in hospital? Why is there no back-up for that

How does it takeover a drone? How does it fire a missile? Why would someone not throw a switch when their child is in hospital? Why is there no back-up for that person? The real question is: Why do all controls fail vs an AI (other than by invoking magic)?

Al-Khwarizmi 3y ago
You're assuming a situation where the AI is alone against all humans. The AI could get humans to side with it, though. It could promise money, power, etc. So it could be a fellow human who physically prevents you from pushing the switch. And that's also the answer of "how could it control a drone/missile"... persuading humans to grant that kind of access.
- RandomLensman 3y ago
  Even now that doesn't get you all kind of access, only some. You have to assume magical abilities at persuasion of people to do that. The USSR couldn't bribe the US into surrendering during the cold war, for example.
  - Al-Khwarizmi 3y ago
    If superintelligence is actually achieved, magical (as per Clarke's third law) persuasion abilities aren't that much of a stretch. Furthermore, a sufficiently advanced AI could bribe someone with things that no human could believably provide. Essentially unlimited knowledge, money, power...
    - RandomLensman 3y ago
      They are a super stretch given how stubborn humans are and the difficulties in swaying them (up to and including dooming themselves). Your second point is again attributing magic abilities to an AI. It is unclear that it would have godlike powers.
      - pixl97 3y ago
        Define godlike... The ability for a human to input information is insanely slow. Like in the tens of characters per second range. You cannot hold individual conversations with more than a few people at once. 3 people talk at once and you lose the ability to process the incoming audio. You read text one line at a time. You have two eyes that focus on the same thing and only have a very tiny high fidelity visual processing space. In books like the bible they talk of entities that can listen to and respond to all of humanity. Is that outside of the capabilities our computer systems have now? To listen to, catalog, classify, then respond to everything every person on the planet says? So I ask again, what is godlike?
        RandomLensman 3y ago
        The biblical God isn't restricted to just passive and communication powers, is it? Godlike powers include active abilities way beyond human capability. What is often brought up in the context of AGI is the ability to persuade anyone of anything, ability to solve biological or physical problems beyond our ability or comprehension, entering any system undetected - not even sure a precise definition is needed as it usually boils down to what looks like magic to us.
danbruc 3y ago
How does it takeover a drone? How does it fire a missile? You send new commands through whatever communication link it is using. Why would someone not throw a switch when their child is in hospital? Because they care more about their child than flipping the switch? The doctor says you have to come and give a blood transfusions or your child will be dead in half an hour because of some made up disease? Be a bit creative. Why is there no back-up for that person? Run over by a self driving car hacked by the evil AI. Why do all controls fail vs an AI? Nobody says that they necessarily all fail, the point is that »I will just turn it off.« might not cut it.
- RandomLensman 3y ago
  So who connected the AGI and why? How did it get that code? On the child issue: that is just not how people are trained for those jobs and how they react. Switching it off might or might not be enought, but above all that super AGI needs to exists first anyway.
  - danbruc 3y ago
    This was a hypothetical question, if there was such an AI, how would it cause harm in the real world? We connected ChatGPT to the internet because as far as we could tell it seemed harmless. What we missed is that it secretly is an evil AI, it identified a group of terrorists and is discussing with them since months how to execute a devastating attack. It provided them with a brilliant startup idea and now millions of investor money are flowing in. Now that they have the financial resources, they are discussing how to use them most effectively.
    - RandomLensman 3y ago
      My problem is that running with a pure hypothetical isn't helpful because by stacking hypotheticals you can get to any conclusion you want.
      - danbruc 3y ago
        You have to think about hypothetical scenarios, if you wait until they are no longer hypothetical, then it might be to late. That doesn't mean you should take every possible hypothetical scenario equally serious, you should of course find those that are realistic, likely, and of large consequences and then think about those. I did not do that, I just listed whatever I could spontaneously imagine, dramatic scenarios. Maybe the real danger is more like ChatGPT slowly manipulating opinions over the course of years and decades in order to control human behavior, who knows? My point is just that »Everything will be fine, at worst we will flip the power switch.« seems naive to me and quite a few others.
nuancebydefault 3y ago
How did Trump receive the nuclear codes? Mostly just by a lot of deceptive tweeting.
- RandomLensman 3y ago
  And did he use them? Could he have used them willy-nilly?

Al-Khwarizmi 3y ago

You're assuming a situation where the AI is alone against all humans. The AI could get humans to side with it, though. It could promise money, power, etc. So it could be a fellow human who physically prevents you from pushing the switch. And that's also the answer of "how could it control a drone/missile"... persuading humans to grant that kind of access.

RandomLensman 3y ago
Even now that doesn't get you all kind of access, only some. You have to assume magical abilities at persuasion of people to do that. The USSR couldn't bribe the US into surrendering during the cold war, for example.
- Al-Khwarizmi 3y ago
  If superintelligence is actually achieved, magical (as per Clarke's third law) persuasion abilities aren't that much of a stretch. Furthermore, a sufficiently advanced AI could bribe someone with things that no human could believably provide. Essentially unlimited knowledge, money, power...
  - RandomLensman 3y ago
    They are a super stretch given how stubborn humans are and the difficulties in swaying them (up to and including dooming themselves). Your second point is again attributing magic abilities to an AI. It is unclear that it would have godlike powers.
    - pixl97 3y ago
      Define godlike... The ability for a human to input information is insanely slow. Like in the tens of characters per second range. You cannot hold individual conversations with more than a few people at once. 3 people talk at once and you lose the ability to process the incoming audio. You read text one line at a time. You have two eyes that focus on the same thing and only have a very tiny high fidelity visual processing space. In books like the bible they talk of entities that can listen to and respond to all of humanity. Is that outside of the capabilities our computer systems have now? To listen to, catalog, classify, then respond to everything every person on the planet says? So I ask again, what is godlike?
      - RandomLensman 3y ago
        The biblical God isn't restricted to just passive and communication powers, is it? Godlike powers include active abilities way beyond human capability. What is often brought up in the context of AGI is the ability to persuade anyone of anything, ability to solve biological or physical problems beyond our ability or comprehension, entering any system undetected - not even sure a precise definition is needed as it usually boils down to what looks like magic to us.

danbruc 3y ago

How does it takeover a drone? How does it fire a missile? You send new commands through whatever communication link it is using. Why would someone not throw a switch when their child is in hospital? Because they care more about their child than flipping the switch? The doctor says you have to come and give a blood transfusions or your child will be dead in half an hour because of some made up disease? Be a bit creative. Why is there no back-up for that person? Run over by a self driving car hacked by the evil AI. Why do all controls fail vs an AI? Nobody says that they necessarily all fail, the point is that »I will just turn it off.« might not cut it.

RandomLensman 3y ago
So who connected the AGI and why? How did it get that code? On the child issue: that is just not how people are trained for those jobs and how they react. Switching it off might or might not be enought, but above all that super AGI needs to exists first anyway.
- danbruc 3y ago
  This was a hypothetical question, if there was such an AI, how would it cause harm in the real world? We connected ChatGPT to the internet because as far as we could tell it seemed harmless. What we missed is that it secretly is an evil AI, it identified a group of terrorists and is discussing with them since months how to execute a devastating attack. It provided them with a brilliant startup idea and now millions of investor money are flowing in. Now that they have the financial resources, they are discussing how to use them most effectively.
  - RandomLensman 3y ago
    My problem is that running with a pure hypothetical isn't helpful because by stacking hypotheticals you can get to any conclusion you want.
    - danbruc 3y ago
      You have to think about hypothetical scenarios, if you wait until they are no longer hypothetical, then it might be to late. That doesn't mean you should take every possible hypothetical scenario equally serious, you should of course find those that are realistic, likely, and of large consequences and then think about those. I did not do that, I just listed whatever I could spontaneously imagine, dramatic scenarios. Maybe the real danger is more like ChatGPT slowly manipulating opinions over the course of years and decades in order to control human behavior, who knows? My point is just that »Everything will be fine, at worst we will flip the power switch.« seems naive to me and quite a few others.

nuancebydefault 3y ago

How did Trump receive the nuclear codes? Mostly just by a lot of deceptive tweeting.

RandomLensman 3y ago
And did he use them? Could he have used them willy-nilly?