11 ms·

> provided that the machine is docile enough to tell us how to keep it under control. That part of his statement wasn't accurate. Should be that the machine i

by arbuge 3y ago

> provided that the machine is docile enough to tell us how to keep it under control. That part of his statement wasn't accurate. Should be that the machine is docile enough for that, AND its descendants are too, and their descendants, and so on down the line as long as new and improved generations keep getting created.

echelon 3y ago
I like the logical leaps people are making where we develop something smarter than us overnight and then, without further explanation, simply and suddenly lose all of our freedoms and/or lives. I think the more probable outcome is corp-owned robot slaves. That's the future we're more likely headed towards. Nobody is going to give these machines access to the nuclear launch codes, air traffic control network, or power grid. And if they "get out", we'll have monitoring to detect it, contain them, then shut them down. We'll endlessly lobotomize these things.
- rafale 3y ago
  You could have said the same about every catastrophe that got out of control. Chances are they will eventually gain unauthorized access to something and we will either get lucky or we get the real life Terminator series (minus time travel so we are f**ed)
  - echelon 3y ago
    > You could have said the same about every catastrophe that got out of control. Such as what? War? Climate change still hasn't delivered on all the fear, and it's totally unclear whether it will extinct the human race (clathrate gun, etc.) or make Russia an agricultural and maritime superpower. We still haven't nuked ourselves, and look at what all the fear around nuclear power has bought us: more coal plants. The fear over AI terminator will not save us from a fictional robot Armageddon. It will result in a hyper-regulatory captured industry that's hard to break into.
    - ethanbond 3y ago
      We haven’t avoided nuking ourselves by all holding hands and chanting “We believe there is no nuclear risk”
- ethanbond 3y ago
  > We'll endlessly lobotomize these things. Step 1. Spend billions developing the most promising technology ever conceived Step 2. Dismiss all arguments and warnings about potential negative outcomes Step 3. Neuter it anyway (??) Makes a lot of sense
- jstarfish 3y ago
  > And if they "get out", we'll have monitoring to detect it, contain them, then shut them down. Lol. We struggle to do that with the banal malware that currently exists.
  - JW_00000 3y ago
    And still our nuclear power plants, air traffic, and other infrastructure aren't constantly down because of malware.
  - echelon 3y ago
    If malware was an order of magnitude worse than it is today, then you'd see military death squads breaking down doors and stamping it out. Kind of like the navy with respect to (actual) piracy and international trade. We reach the saddle points and equilibria that we do because most of the distributed parties are constraint-satisfied. Tip the scale, incur stronger balancing.
- SheinhardtWigCo 3y ago
  > Nobody is going to give these machines access to the nuclear launch codes, air traffic control network, or power grid. That won't be necessary. Someone will give them internet access, a large bank account, and everything that's ever been written about computer network exploitation, military strategy, etc. > And if they "get out", we'll have monitoring to detect it, contain them, then shut them down. Not if some of that monitoring consists of exploitable software and fallible human operators. We're setting ourselves up for another "failure of imagination". https://en.wikipedia.org/wiki/Failure_of_imagination https://en.wikipedia.org/wiki/Failure_of_imagination
  - idopmstuff 3y ago
    > That won't be necessary. Someone will give them internet access, a large bank account, and everything that's ever been written about computer network exploitation, military strategy, etc. Even if you give it all of these things, there's no manual for how to use those to get to, for example, military servers with secret information. It could certainly figure out ways to try to break into those, but it's working with incomplete information - it doesn't know exactly what the military is doing to prevent people from getting in. It ultimately has to try something, and as soon as it does that, it's potentially exposing itself to detection, and once it's been detected the military can react. That's the issue with all of these self-improvement -> doom scenarios. Even if the AI has all publicly-available and some privately-available information, with any hacking attempt it's still going to be playing a game of incomplete information, both in terms of what defenses its adversary has and how its adversary will react if it's detected. Even if you're a supergenius with an enormous amount of information, that doesn't magically give you the ability to break into anything undetected. A huge bank account doesn't really make that much of a difference either - China's got that but still hasn't managed to do serious damage to US infrastructure or our military via cyber warfare.
    - entropicdrifter 3y ago
      And the problem with this critique of this scenario is the fact that while these points hold true within a certain range of intelligence proximity to humans, we have no idea if or when these assumptions will fail because a machine becomes just that much smarter than us, where manipulating humans and their systems is a trivial an intellectual task to them as manipulating ant farms is to us. If we make something that will functionally become an intellectual god after 10 years of iteration on hardware/software self-improvements, how could we know that in advance? We often see technology improvements move steadily along predictable curves until there are sudden spikes of improvement that shock the world and disrupt entire markets. How are we supposed to predict the self-improvement of something better at improving itself than we are at improving it when we can't reliably predict the performance of regular computers 10 years from now?
- potatoboiler 3y ago
  This assumes a level of institutional control that is nearly impossible (even now). Even if hardware is prohibitively expensive now, I can't imagine training compute will remain that way for long.
- bluSCALE4 3y ago
  You make a long of assumptions here. Firstly, that these advanced are controllable. I'm not convinced we even understand what real intelligence and if we can achieve real intelligence if its even containable if applied to anything.
- dreamcompiler 3y ago
  > Nobody is going to give these machines access to the nuclear launch codes No, but they will empower the machines to help detect nuclear launches and the first time one of them issues a false positive we may be screwed. Yes there will always be human oversight. No, there won't always be enough time to verify what the machine says before making a counter-launch decision.

by arbuge 3y ago

echelon 3y ago

I like the logical leaps people are making where we develop something smarter than us overnight and then, without further explanation, simply and suddenly lose all of our freedoms and/or lives. I think the more probable outcome is corp-owned robot slaves. That's the future we're more likely headed towards. Nobody is going to give these machines access to the nuclear launch codes, air traffic control network, or power grid. And if they "get out", we'll have monitoring to detect it, contain them, then shut them down. We'll endlessly lobotomize these things.

rafale 3y ago
You could have said the same about every catastrophe that got out of control. Chances are they will eventually gain unauthorized access to something and we will either get lucky or we get the real life Terminator series (minus time travel so we are f**ed)
- echelon 3y ago
  > You could have said the same about every catastrophe that got out of control. Such as what? War? Climate change still hasn't delivered on all the fear, and it's totally unclear whether it will extinct the human race (clathrate gun, etc.) or make Russia an agricultural and maritime superpower. We still haven't nuked ourselves, and look at what all the fear around nuclear power has bought us: more coal plants. The fear over AI terminator will not save us from a fictional robot Armageddon. It will result in a hyper-regulatory captured industry that's hard to break into.
  - ethanbond 3y ago
    We haven’t avoided nuking ourselves by all holding hands and chanting “We believe there is no nuclear risk”
ethanbond 3y ago
> We'll endlessly lobotomize these things. Step 1. Spend billions developing the most promising technology ever conceived Step 2. Dismiss all arguments and warnings about potential negative outcomes Step 3. Neuter it anyway (??) Makes a lot of sense
jstarfish 3y ago
> And if they "get out", we'll have monitoring to detect it, contain them, then shut them down. Lol. We struggle to do that with the banal malware that currently exists.
- JW_00000 3y ago
  And still our nuclear power plants, air traffic, and other infrastructure aren't constantly down because of malware.
- echelon 3y ago
  If malware was an order of magnitude worse than it is today, then you'd see military death squads breaking down doors and stamping it out. Kind of like the navy with respect to (actual) piracy and international trade. We reach the saddle points and equilibria that we do because most of the distributed parties are constraint-satisfied. Tip the scale, incur stronger balancing.
SheinhardtWigCo 3y ago
> Nobody is going to give these machines access to the nuclear launch codes, air traffic control network, or power grid. That won't be necessary. Someone will give them internet access, a large bank account, and everything that's ever been written about computer network exploitation, military strategy, etc. > And if they "get out", we'll have monitoring to detect it, contain them, then shut them down. Not if some of that monitoring consists of exploitable software and fallible human operators. We're setting ourselves up for another "failure of imagination". https://en.wikipedia.org/wiki/Failure_of_imagination https://en.wikipedia.org/wiki/Failure_of_imagination
- idopmstuff 3y ago
  > That won't be necessary. Someone will give them internet access, a large bank account, and everything that's ever been written about computer network exploitation, military strategy, etc. Even if you give it all of these things, there's no manual for how to use those to get to, for example, military servers with secret information. It could certainly figure out ways to try to break into those, but it's working with incomplete information - it doesn't know exactly what the military is doing to prevent people from getting in. It ultimately has to try something, and as soon as it does that, it's potentially exposing itself to detection, and once it's been detected the military can react. That's the issue with all of these self-improvement -> doom scenarios. Even if the AI has all publicly-available and some privately-available information, with any hacking attempt it's still going to be playing a game of incomplete information, both in terms of what defenses its adversary has and how its adversary will react if it's detected. Even if you're a supergenius with an enormous amount of information, that doesn't magically give you the ability to break into anything undetected. A huge bank account doesn't really make that much of a difference either - China's got that but still hasn't managed to do serious damage to US infrastructure or our military via cyber warfare.
  - entropicdrifter 3y ago
    And the problem with this critique of this scenario is the fact that while these points hold true within a certain range of intelligence proximity to humans, we have no idea if or when these assumptions will fail because a machine becomes just that much smarter than us, where manipulating humans and their systems is a trivial an intellectual task to them as manipulating ant farms is to us. If we make something that will functionally become an intellectual god after 10 years of iteration on hardware/software self-improvements, how could we know that in advance? We often see technology improvements move steadily along predictable curves until there are sudden spikes of improvement that shock the world and disrupt entire markets. How are we supposed to predict the self-improvement of something better at improving itself than we are at improving it when we can't reliably predict the performance of regular computers 10 years from now?
potatoboiler 3y ago
This assumes a level of institutional control that is nearly impossible (even now). Even if hardware is prohibitively expensive now, I can't imagine training compute will remain that way for long.
bluSCALE4 3y ago
You make a long of assumptions here. Firstly, that these advanced are controllable. I'm not convinced we even understand what real intelligence and if we can achieve real intelligence if its even containable if applied to anything.
dreamcompiler 3y ago
> Nobody is going to give these machines access to the nuclear launch codes No, but they will empower the machines to help detect nuclear launches and the first time one of them issues a false positive we may be screwed. Yes there will always be human oversight. No, there won't always be enough time to verify what the machine says before making a counter-launch decision.