4 ms·

How does it takeover a drone? How does it fire a missile? You send new commands through whatever communication link it is using. Why would someone not throw a

How does it takeover a drone? How does it fire a missile? You send new commands through whatever communication link it is using. Why would someone not throw a switch when their child is in hospital? Because they care more about their child than flipping the switch? The doctor says you have to come and give a blood transfusions or your child will be dead in half an hour because of some made up disease? Be a bit creative. Why is there no back-up for that person? Run over by a self driving car hacked by the evil AI. Why do all controls fail vs an AI? Nobody says that they necessarily all fail, the point is that »I will just turn it off.« might not cut it.

RandomLensman 3y ago
So who connected the AGI and why? How did it get that code? On the child issue: that is just not how people are trained for those jobs and how they react. Switching it off might or might not be enought, but above all that super AGI needs to exists first anyway.
- danbruc 3y ago
  This was a hypothetical question, if there was such an AI, how would it cause harm in the real world? We connected ChatGPT to the internet because as far as we could tell it seemed harmless. What we missed is that it secretly is an evil AI, it identified a group of terrorists and is discussing with them since months how to execute a devastating attack. It provided them with a brilliant startup idea and now millions of investor money are flowing in. Now that they have the financial resources, they are discussing how to use them most effectively.
  - RandomLensman 3y ago
    My problem is that running with a pure hypothetical isn't helpful because by stacking hypotheticals you can get to any conclusion you want.
    - danbruc 3y ago
      You have to think about hypothetical scenarios, if you wait until they are no longer hypothetical, then it might be to late. That doesn't mean you should take every possible hypothetical scenario equally serious, you should of course find those that are realistic, likely, and of large consequences and then think about those. I did not do that, I just listed whatever I could spontaneously imagine, dramatic scenarios. Maybe the real danger is more like ChatGPT slowly manipulating opinions over the course of years and decades in order to control human behavior, who knows? My point is just that »Everything will be fine, at worst we will flip the power switch.« seems naive to me and quite a few others.
      - RandomLensman 3y ago
        We by-and-large ignore a lot hypothetical (and non-hypothetical) risks, what I am missing is a sound argument why this one deserves particular attention vs the others. Otherwise, "just flip the switch" is a potential non-naive approach given limited resources.
        danbruc 3y ago
        Flip the switch is the naive approach because it assumes - or hopes - that you can. It is as naive as saying that if you accidentally touch an electrical wire and get shocked, just let go or flip the switch, problem solved. That is ignorant of the fact that touching an electrical wire might make it impossible to let go because you loose control over your muscles and it fails to consider that a switch might be out of reach. We know from countless experiments that AIs can behave in unexpected ways. We know how indifferent we humans can be to other life. We are pretty careful when we approach unknown lifeforms, whether microbes in the lab or animals in the dschungel. We would probably be pretty careful towards aliens. We have also not been careful with new thing, for example when we discovered radioactivity, and it certainly caused harm. I do not see from where we should get justification for an attitude towards AI that there are no risks worth considering.
        RandomLensman 3y ago

RandomLensman 3y ago

So who connected the AGI and why? How did it get that code? On the child issue: that is just not how people are trained for those jobs and how they react. Switching it off might or might not be enought, but above all that super AGI needs to exists first anyway.

danbruc 3y ago
This was a hypothetical question, if there was such an AI, how would it cause harm in the real world? We connected ChatGPT to the internet because as far as we could tell it seemed harmless. What we missed is that it secretly is an evil AI, it identified a group of terrorists and is discussing with them since months how to execute a devastating attack. It provided them with a brilliant startup idea and now millions of investor money are flowing in. Now that they have the financial resources, they are discussing how to use them most effectively.
- RandomLensman 3y ago
  My problem is that running with a pure hypothetical isn't helpful because by stacking hypotheticals you can get to any conclusion you want.
  - danbruc 3y ago
    You have to think about hypothetical scenarios, if you wait until they are no longer hypothetical, then it might be to late. That doesn't mean you should take every possible hypothetical scenario equally serious, you should of course find those that are realistic, likely, and of large consequences and then think about those. I did not do that, I just listed whatever I could spontaneously imagine, dramatic scenarios. Maybe the real danger is more like ChatGPT slowly manipulating opinions over the course of years and decades in order to control human behavior, who knows? My point is just that »Everything will be fine, at worst we will flip the power switch.« seems naive to me and quite a few others.
    - RandomLensman 3y ago
      We by-and-large ignore a lot hypothetical (and non-hypothetical) risks, what I am missing is a sound argument why this one deserves particular attention vs the others. Otherwise, "just flip the switch" is a potential non-naive approach given limited resources.
      - danbruc 3y ago
        Flip the switch is the naive approach because it assumes - or hopes - that you can. It is as naive as saying that if you accidentally touch an electrical wire and get shocked, just let go or flip the switch, problem solved. That is ignorant of the fact that touching an electrical wire might make it impossible to let go because you loose control over your muscles and it fails to consider that a switch might be out of reach. We know from countless experiments that AIs can behave in unexpected ways. We know how indifferent we humans can be to other life. We are pretty careful when we approach unknown lifeforms, whether microbes in the lab or animals in the dschungel. We would probably be pretty careful towards aliens. We have also not been careful with new thing, for example when we discovered radioactivity, and it certainly caused harm. I do not see from where we should get justification for an attitude towards AI that there are no risks worth considering.
        RandomLensman 3y ago