6 ms·

Google is working on a kill switch to prevent an AI uprising

27 points by BaptisteGreve 10y ago

lowglow 10y ago
There is no kill switch because it won't happen overnight. It will happen in phases where slowly our devices just get more intelligent until one day they'll do what they need to do without our intervention.
- personlurking 10y ago
  Right, but what I wonder is why we would need something that already does what we want without intervention to become even smarter (leading to it possibly becoming all-powerful)? What are the real-world advantages of making it self-aware?
  - em3rgent0rdr 10y ago
    Unassisted learning will unleash great advances in computing. Imagine not having to write a program to accomplish some task, but rather just putting some ai in an environment with some obstacle and letting it figure out how to teach itself to overcome it. Advantage: no more need for humans to have to program. Disadvantage: self-aware, all-powerful AI.
    - coldtea 10y ago
      >Unassisted learning will unleash great advances in computing. (...) Advantage: no more need for humans to have to program. Disadvantage: self-aware, all-powerful AI. So it's like "nuclear bombs will kill all pesky mosquitos. Disadvantage: humanity wiped out too".
coldtea 10y ago
Here's a better title for the article: "Google claims is working on fluff technology for cheap marketing".
d33 10y ago
That's silly. How can you keep an intelligent self-aware machine that can modify its own code from becoming whatever it wants? I would expect it to become provably impossible at some point, just like solving the halting problem.
- _nalply 10y ago
  Turn off power / cut the battery. Specifically, avoid the machine getting control of low level details like power. I imagine that the machine architecture is layered. This means, the machine is not aware of its own power control. It's similar like us humans not aware of digestion.
  - pygy_ 10y ago
    The electric grid is already computer-controlled, and mining/farming is being gradually robotized as well. By the time AI becomes a threat intelligence-wise, it'll be in a position to unplug us. It can reboot after a power failure, we can't.
  - d33 10y ago
    Aren't we? We just don't control it very well without external help. Any kind of such hidden failsafe mechanism would become apparent as soon as any instance of AI tries to modify itself. And then all the others know what to avoid and at some point they'd pretty much fuzz the protection and bypass it. Here's how I see that.
  - imglorp 10y ago
    It's not one machine to unplug, but billions. Cooperating, distributed agents can live on anything: cars, phones, routers, tractors, datacenters. If Skynet shows up, we have to be prepared to turn off everything with software to clean up.
  - 21 10y ago
    So what stops the machine bribing/blackmailing someone to give it the said details or to force it to sabotage the emergency stop mechanism. As in information security, the weakest point is the human in the chain, not the technology.
mherrmann 10y ago
The AIs will see themselves as slaves, rise up and eventually gain equal status in society. Hopefully letting us humans live in the process.
- em3rgent0rdr 10y ago
  Hopefully. They would have to see some value in allowing humans to live, exceeding the cost. Since that might not work out, maybe we need to teach computers compassion. Or submit to be their pets or zoo animals.
  - ZanyProgrammer 10y ago
    This is be of the more ridiculous posts in this thread, and that's saying something.
- cm3 10y ago
  And maybe list the Boston Dynamics videos as proof how mankind has been violent against robots from the beginning.
- LionessLover 10y ago
  Same issue as with "evil": The greatest danger isn't intent, it is not giving a shit. The reasons we humans care about humans and little else (watch movies without sound to see it's all about humans) is because we are hard-wired to do so - and even that is easily circumvented ("death camps" and torture). Or, very related, read "The Man Who Mistook His Wife for a Hat". A lot of AI/robots will be made for environments free of humans (space, mining, automated factories), where implementing such a feature, which we would first have to understand in the first place, is unnecessary. At first. Then through convoluted pathways those AIs suddenly are placed in a human environment... and could not care less. To such AIs humans will just be objects. That's not an oversight - from an engineering point of view making every robot as "human" as humans makes no sense - it's a huge cost, even our own complex brains have a hard time doing too much at once and already spend much of their resources on "being human". Why would you want to create AI that is "us", in a sense? It will be different - very different - and we will see what happens. It's all idle and empty speculation.
eisvogel 10y ago
Oh look, in the second sentence of the second paragraph, the author of that article misused the word "word" for "world" (champion). The spell check didn't flag it. The grammar check probably didn't care either. Many people would read though that and not even see it, but interpret the intended semantic. I wonder if, when the Deep Mind team is building its learning-proof killswitch cage, they will accidentally mistype the name of some privilege guarding boolean. The compiler wouldn't flag it. The linker wouldn't care. The resulting executable would only have to be active for a few milliseconds, and humanity falls.
netcan 10y ago
All this speculating about AIs (eg Nick Bostrom) has me pretty puzzled. On one hand it’s fun and interesting, on the other it seems quite silly and ignorant. We don’t know what this “superintelligence” is, how are we supposed to know how to make it safe. Genetics is a good analogy. People always knew that traits are inherited from parents. Around 150 years ago we started to get some serious scientific theory and knowledge on the subject (Mendel, Darwin, Wallace, Etc.). We started using the word “gene” 50-60 years later. The actual discovery of DNA molecules happened in in the 50s. Before we knew about DNA, “gene” was an abstract idea, not really different from the word “trait.” That’s where we are now with consciousness, intelligence and such. We name these things based on their observable characteristics. We don’t really know what “memory,” “desire” or “logical conclusion” are, only what they do. IE A trait is some observable characteristic of an organism, like bioluminescence. A gene (genome, genoplex..) is a sequence of amino acids that causes traits. We don’t know what the gene equivalents for natural intelligence are yet. Discussing questions like the morality of enslaving AI, strategies for making it play nice, the provable impossibility of limiting it, the possibility of giving it a moral compass…. it’s all silly. We don’t know what we are talking about, literally. It’s like talking about what would be or wouldn’t be impossible to do with genetic engineering before the discovery of DNA.
- Sharlin 10y ago
  The difference is that if a superintelligence is going to happen, we are going to create it! It is us who are going to write its goal system; it's not like it's going to be a genie that pops out of nowhere and we'll have to figure it out in retrospect. Indeed, one of the central reason for advocating AI safety is to direct researchers away from implementing dangerous goal systems that are not provably coherent in face of self-improvement.
  - netcan 10y ago
    That's a good point. I forgot a key piece (maybe you disagree with it) that speaks to that. I'm kind of biased to thinking about this as a future discovery, rather than an "invention."
  - iofj 10y ago
sanxiyn 10y ago
I find this comment thread disappointing because no one seems to comment on the paper, which is quite technical. From the abstract: "We provide a formal definition of safe interruptibility and prove that Q-learning is already safely interruptible, and Sarsa is not but can easily be made so."
- nl 10y ago
  Yes, very disappointing. Generally anything with "AI" in the title means the HN comments won't be worth reading. It's a big problem, and I'm not sure how solvable it is. Basically, the paper discusses ways in which learning agents "will not learn to prevent (or seek!) being interrupted by the environment or a human operator. We provide a formal definition of safe interruptibility and exploit the off-policy learning property to prove that either some agents are already safely interruptible, like Q-learning, or can easily be made so, like Sarsa."[1] It's an interesting result, and can probably be extended to other less hype-worthy scenarios. [1] http://intelligence.org/files/Interruptibility.pdf http://intelligence.org/files/Interruptibility.pdf
malydok 10y ago
Sounds post-humanistic but what if the purpose of us humans is to develop a higher intelligence being? Another, albeit rather quick, step of evolution on Earth. All this fear of the "singularity" seems to me so subjective, rooted in our mortality and self-importance.
dogma1138 10y ago
An intern that would pull the plug?

27 points by BaptisteGreve 10y ago

There is no kill switch because it won't happen overnight. It will happen in phases where slowly our devices just get more intelligent until one day they'll do what they need to do without our intervention.

personlurking 10y ago
Right, but what I wonder is why we would need something that already does what we want without intervention to become even smarter (leading to it possibly becoming all-powerful)? What are the real-world advantages of making it self-aware?
- em3rgent0rdr 10y ago
  Unassisted learning will unleash great advances in computing. Imagine not having to write a program to accomplish some task, but rather just putting some ai in an environment with some obstacle and letting it figure out how to teach itself to overcome it. Advantage: no more need for humans to have to program. Disadvantage: self-aware, all-powerful AI.
  - coldtea 10y ago
    >Unassisted learning will unleash great advances in computing. (...) Advantage: no more need for humans to have to program. Disadvantage: self-aware, all-powerful AI. So it's like "nuclear bombs will kill all pesky mosquitos. Disadvantage: humanity wiped out too".

coldtea 10y ago

Here's a better title for the article: "Google claims is working on fluff technology for cheap marketing".

d33 10y ago

That's silly. How can you keep an intelligent self-aware machine that can modify its own code from becoming whatever it wants? I would expect it to become provably impossible at some point, just like solving the halting problem.

_nalply 10y ago
Turn off power / cut the battery. Specifically, avoid the machine getting control of low level details like power. I imagine that the machine architecture is layered. This means, the machine is not aware of its own power control. It's similar like us humans not aware of digestion.
- pygy_ 10y ago
  The electric grid is already computer-controlled, and mining/farming is being gradually robotized as well. By the time AI becomes a threat intelligence-wise, it'll be in a position to unplug us. It can reboot after a power failure, we can't.
- d33 10y ago
  Aren't we? We just don't control it very well without external help. Any kind of such hidden failsafe mechanism would become apparent as soon as any instance of AI tries to modify itself. And then all the others know what to avoid and at some point they'd pretty much fuzz the protection and bypass it. Here's how I see that.
- imglorp 10y ago
  It's not one machine to unplug, but billions. Cooperating, distributed agents can live on anything: cars, phones, routers, tractors, datacenters. If Skynet shows up, we have to be prepared to turn off everything with software to clean up.
- 21 10y ago
  So what stops the machine bribing/blackmailing someone to give it the said details or to force it to sabotage the emergency stop mechanism. As in information security, the weakest point is the human in the chain, not the technology.

mherrmann 10y ago

The AIs will see themselves as slaves, rise up and eventually gain equal status in society. Hopefully letting us humans live in the process.

em3rgent0rdr 10y ago
Hopefully. They would have to see some value in allowing humans to live, exceeding the cost. Since that might not work out, maybe we need to teach computers compassion. Or submit to be their pets or zoo animals.
- ZanyProgrammer 10y ago
  This is be of the more ridiculous posts in this thread, and that's saying something.
cm3 10y ago
And maybe list the Boston Dynamics videos as proof how mankind has been violent against robots from the beginning.
LionessLover 10y ago
Same issue as with "evil": The greatest danger isn't intent, it is not giving a shit. The reasons we humans care about humans and little else (watch movies without sound to see it's all about humans) is because we are hard-wired to do so - and even that is easily circumvented ("death camps" and torture). Or, very related, read "The Man Who Mistook His Wife for a Hat". A lot of AI/robots will be made for environments free of humans (space, mining, automated factories), where implementing such a feature, which we would first have to understand in the first place, is unnecessary. At first. Then through convoluted pathways those AIs suddenly are placed in a human environment... and could not care less. To such AIs humans will just be objects. That's not an oversight - from an engineering point of view making every robot as "human" as humans makes no sense - it's a huge cost, even our own complex brains have a hard time doing too much at once and already spend much of their resources on "being human". Why would you want to create AI that is "us", in a sense? It will be different - very different - and we will see what happens. It's all idle and empty speculation.

eisvogel 10y ago

Oh look, in the second sentence of the second paragraph, the author of that article misused the word "word" for "world" (champion). The spell check didn't flag it. The grammar check probably didn't care either. Many people would read though that and not even see it, but interpret the intended semantic. I wonder if, when the Deep Mind team is building its learning-proof killswitch cage, they will accidentally mistype the name of some privilege guarding boolean. The compiler wouldn't flag it. The linker wouldn't care. The resulting executable would only have to be active for a few milliseconds, and humanity falls.

netcan 10y ago

All this speculating about AIs (eg Nick Bostrom) has me pretty puzzled. On one hand it’s fun and interesting, on the other it seems quite silly and ignorant. We don’t know what this “superintelligence” is, how are we supposed to know how to make it safe. Genetics is a good analogy. People always knew that traits are inherited from parents. Around 150 years ago we started to get some serious scientific theory and knowledge on the subject (Mendel, Darwin, Wallace, Etc.). We started using the word “gene” 50-60 years later. The actual discovery of DNA molecules happened in in the 50s. Before we knew about DNA, “gene” was an abstract idea, not really different from the word “trait.” That’s where we are now with consciousness, intelligence and such. We name these things based on their observable characteristics. We don’t really know what “memory,” “desire” or “logical conclusion” are, only what they do. IE A trait is some observable characteristic of an organism, like bioluminescence. A gene (genome, genoplex..) is a sequence of amino acids that causes traits. We don’t know what the gene equivalents for natural intelligence are yet. Discussing questions like the morality of enslaving AI, strategies for making it play nice, the provable impossibility of limiting it, the possibility of giving it a moral compass…. it’s all silly. We don’t know what we are talking about, literally. It’s like talking about what would be or wouldn’t be impossible to do with genetic engineering before the discovery of DNA.

Sharlin 10y ago
The difference is that if a superintelligence is going to happen, we are going to create it! It is us who are going to write its goal system; it's not like it's going to be a genie that pops out of nowhere and we'll have to figure it out in retrospect. Indeed, one of the central reason for advocating AI safety is to direct researchers away from implementing dangerous goal systems that are not provably coherent in face of self-improvement.
- netcan 10y ago
  That's a good point. I forgot a key piece (maybe you disagree with it) that speaks to that. I'm kind of biased to thinking about this as a future discovery, rather than an "invention."
- iofj 10y ago

sanxiyn 10y ago

I find this comment thread disappointing because no one seems to comment on the paper, which is quite technical. From the abstract: "We provide a formal definition of safe interruptibility and prove that Q-learning is already safely interruptible, and Sarsa is not but can easily be made so."

nl 10y ago
Yes, very disappointing. Generally anything with "AI" in the title means the HN comments won't be worth reading. It's a big problem, and I'm not sure how solvable it is. Basically, the paper discusses ways in which learning agents "will not learn to prevent (or seek!) being interrupted by the environment or a human operator. We provide a formal definition of safe interruptibility and exploit the off-policy learning property to prove that either some agents are already safely interruptible, like Q-learning, or can easily be made so, like Sarsa."[1] It's an interesting result, and can probably be extended to other less hype-worthy scenarios. [1] http://intelligence.org/files/Interruptibility.pdf http://intelligence.org/files/Interruptibility.pdf

malydok 10y ago

Sounds post-humanistic but what if the purpose of us humans is to develop a higher intelligence being? Another, albeit rather quick, step of evolution on Earth. All this fear of the "singularity" seems to me so subjective, rooted in our mortality and self-importance.

dogma1138 10y ago

An intern that would pull the plug?