4 ms·

So a sort-of-on-topic question: for all the people who are warning of the coming AI apocalypse and warning us to be careful with AI... how do you enforce that?

So a sort-of-on-topic question: for all the people who are warning of the coming AI apocalypse and warning us to be careful with AI... how do you enforce that? What’s the enforcement mechanism to ensure that the bad people don’t get AI and that people don’t create bad AI? All these warnings, but what can we do about it? If strong AI is invented, especially if it’s open-sourced, the cat is out of the bag and it’s game over, right? We’ve seen how good we are at preventing rogue states from getting nukes. How can we keep them from getting life-ending AI?

emerged 9y ago
I think it's going to parallel the virus/anti-virus eternal struggle. The bad AI will need to be targeted by the good. It'll be a mess, of course.
- ericb 9y ago
  I think any breakaway AI would be a highlander situation in that it would ensure there can be only one.
  - sharemywin 9y ago
    We already created super AI. It's called the stock market. Look at how it's tricked the richest(capitalist) and smartest via engineering jobs into toiling away at getting a number to go up.
  - freehunter 9y ago
    Yeah, that's more of my point, you're right. If we're considering the endgame of the AI singularity, only one AI can win. If there are multiple, they'll each keep getting better, and they'll each keep warring against the other one until only one is left (or none are left). That's the doomsday scenario for AI when people warn that this is coming. But out of all the people warning about it, I haven't yet seen anyone say "this is how we prevent it".
    - optimuspaul 9y ago
      Seems like the only prevention proposed is abstinence and research shows that abstinence doesn't work. In fact telling most people not to do something makes them want to do it more. In fact I'm struggling right now to make my own attempts.
    - ericb 9y ago
      I think the only way to win is to make sure the first one is perfectly friendly, or to make sure we are integrated with the first one. I don't give us great odds on either front.
      - freehunter 9y ago
        I'm trying to remember a book I read a while back, it was a sci-fi book where in the beginning chapters a war was being fought between the US and China to prevent China from developing a strong AI. The main character is killed but the preserved his body and brought him back to life some years later in a cyborg body, where he learns that the world is entirely controlled by a friendly benevolent AI and everyone has it embedded in their skin. But the AI goes rogue suddenly and the only people who survive are the luddites who refused to join the AI and the people who were off-world at the time the AI went rogue. For the life of me I can't find the book even through months of Google searching, but anyway my point is... a "friendly" AI can be hijacked. Or decide to go rogue, since we can't begin to comprehend its thought processes. --edit: sorry I went off topic a bit... I don't want to be just another one of those "AI will kill us all" kinds of people because, as my first comment suggested, that's completely useless fearmongering unless someone has a way to stop it. Kind of like saying "the supervolcano" or "solar flares" or something else completely unpreventable will kill us all... unless there's something you can do to stop it, life goes on until it doesn't. I'd be interested in hearing how to stop it, otherwise I wish people would stop printing these articles.
        ericb 9y ago
        I follow the point you're making. We do have options and perhaps if we were worried enough, they'd be put into effect. You could make AI research illegal. You could place massive R&D to human/brain interfaces. You could remove or reduce the medical device testing restrictions on the basis that this is an all-or-nothing bet for humanity.
akvadrako 9y ago
It's clear to me that if AI exists it will be used irresponsibly and by bad actors. So the only way to be safe is prevent all research along those lines for as long as possible†. The focus for now should be on getting humanity to work together and getting a firm grasp on ethics/morality to see how AI could be convinced to be good, instead of enforced. † As in the background story of Dune.
anvandare 9y ago
There is none. Science is discovering what is possible. Technology is changing something from possible to doable. Once the possibility of something is made clear (if not the exact path toward it - the do-ability), people will start rushing for it - if only out of fear that someone else might have it while they themselves do not. At least nuclear technology has the downside of being very expensive and large-scale (so few players) to produce, and being pretty obvious when it's actually used. Other things that are becoming increasingly possible (but still not doable) - like modified viruses that only target people with a specific DNA profile - will be a lot easier to hide, both their production and their usage. Same thing with AI.
Symmetry 9y ago
Well, here's one approach: The Machine Intelligence Research Institute is a research nonprofit studying the mathematical underpinnings of intelligent behavior. Our mission is to develop formal tools for the clean design and analysis of general-purpose AI systems, with the intent of making such systems safer and more reliable when they are developed. https://intelligence.org/about/ https://intelligence.org/about/ That doesn't help with people developing AI with bad intentions but the Sorcerer's Apprentice problem is important too.

emerged 9y ago

I think it's going to parallel the virus/anti-virus eternal struggle. The bad AI will need to be targeted by the good. It'll be a mess, of course.

ericb 9y ago
I think any breakaway AI would be a highlander situation in that it would ensure there can be only one.
- sharemywin 9y ago
  We already created super AI. It's called the stock market. Look at how it's tricked the richest(capitalist) and smartest via engineering jobs into toiling away at getting a number to go up.
- freehunter 9y ago
  Yeah, that's more of my point, you're right. If we're considering the endgame of the AI singularity, only one AI can win. If there are multiple, they'll each keep getting better, and they'll each keep warring against the other one until only one is left (or none are left). That's the doomsday scenario for AI when people warn that this is coming. But out of all the people warning about it, I haven't yet seen anyone say "this is how we prevent it".
  - optimuspaul 9y ago
    Seems like the only prevention proposed is abstinence and research shows that abstinence doesn't work. In fact telling most people not to do something makes them want to do it more. In fact I'm struggling right now to make my own attempts.
  - ericb 9y ago
    I think the only way to win is to make sure the first one is perfectly friendly, or to make sure we are integrated with the first one. I don't give us great odds on either front.
    - freehunter 9y ago
      I'm trying to remember a book I read a while back, it was a sci-fi book where in the beginning chapters a war was being fought between the US and China to prevent China from developing a strong AI. The main character is killed but the preserved his body and brought him back to life some years later in a cyborg body, where he learns that the world is entirely controlled by a friendly benevolent AI and everyone has it embedded in their skin. But the AI goes rogue suddenly and the only people who survive are the luddites who refused to join the AI and the people who were off-world at the time the AI went rogue. For the life of me I can't find the book even through months of Google searching, but anyway my point is... a "friendly" AI can be hijacked. Or decide to go rogue, since we can't begin to comprehend its thought processes. --edit: sorry I went off topic a bit... I don't want to be just another one of those "AI will kill us all" kinds of people because, as my first comment suggested, that's completely useless fearmongering unless someone has a way to stop it. Kind of like saying "the supervolcano" or "solar flares" or something else completely unpreventable will kill us all... unless there's something you can do to stop it, life goes on until it doesn't. I'd be interested in hearing how to stop it, otherwise I wish people would stop printing these articles.
      - ericb 9y ago
        I follow the point you're making. We do have options and perhaps if we were worried enough, they'd be put into effect. You could make AI research illegal. You could place massive R&D to human/brain interfaces. You could remove or reduce the medical device testing restrictions on the basis that this is an all-or-nothing bet for humanity.

akvadrako 9y ago

It's clear to me that if AI exists it will be used irresponsibly and by bad actors. So the only way to be safe is prevent all research along those lines for as long as possible†. The focus for now should be on getting humanity to work together and getting a firm grasp on ethics/morality to see how AI could be convinced to be good, instead of enforced. † As in the background story of Dune.

anvandare 9y ago

There is none. Science is discovering what is possible. Technology is changing something from possible to doable. Once the possibility of something is made clear (if not the exact path toward it - the do-ability), people will start rushing for it - if only out of fear that someone else might have it while they themselves do not. At least nuclear technology has the downside of being very expensive and large-scale (so few players) to produce, and being pretty obvious when it's actually used. Other things that are becoming increasingly possible (but still not doable) - like modified viruses that only target people with a specific DNA profile - will be a lot easier to hide, both their production and their usage. Same thing with AI.

Symmetry 9y ago

Well, here's one approach: The Machine Intelligence Research Institute is a research nonprofit studying the mathematical underpinnings of intelligent behavior. Our mission is to develop formal tools for the clean design and analysis of general-purpose AI systems, with the intent of making such systems safer and more reliable when they are developed. https://intelligence.org/about/ https://intelligence.org/about/ That doesn't help with people developing AI with bad intentions but the Sorcerer's Apprentice problem is important too.