5 ms·

"I think the fears about “evil killer robots” are overblown. There’s a big difference between intelligence and sentience. Our software is becoming more intellig

by rrggrr 12y ago

"I think the fears about “evil killer robots” are overblown. There’s a big difference between intelligence and sentience. Our software is becoming more intelligent, but that does not imply it is about to become sentient." It is the period between sentience and "advanced" artificial intelligence that should be worrying for two reasons: First, it is close at hand. More importantly, the unintended consequences of tech are never well thought out in the initial stages of adoption. I'm reading Eric Schlosser's book on the early days of nuclear weapons and the many, many near misses the US experienced as it adopted nuclear arms without much thought to risk management. I see parallels in the race to develop and deploy pre-sentient A.I. Link below to Schlosser's book. http://www.amazon.com/Command-Control-Damascus-Accident-Illusion/dp/0143125788/ref=sr_1_1?ie=UTF8&qid=1422904872&sr=8-1&keywords=nuclear+command+safety http://www.amazon.com/Command-Control-Damascus-Accident-Illu...

chubot 12y ago
I've been thinking about what I visualize as "plant" intelligence, in response to Elon Musk's statements about killer AI. It seems like Andrew Ng is saying the same thing. The metaphor is that you could have "sedentary" superintelligences that solve a relatively narrow problem. They take in huge amounts of data, and spit out ingenious answers to well-defined problems. Things like traffic-routing, energy allocation, or photo/video recognition, etc. They are tools for humans. By what mechanism do these superintelligences involve into organisms motivated to extinguish humans? Reproduction, which leads to independent evolution. That process is inherently unpredictable. But right now we're firmly in the stage of humans creating AI. Our AI is nowhere near capable enough to create more AI. And I would argue that there is no possibility of this happening by "accident"; you would have to specifically engineer AI to able to create more AI. I agree with Ng in that it's more of a distraction now than anything else. The jobs issue is a lot more pertinent. It reminds me of this book I got at Google 8 or 9 years ago. This guy asked a robotic professors how they would avoid being killed by a robot. And the professor said: "Climb up one step" (on a staircase).
- gojomo 12y ago
  "Our AI is nowhere near capable enough to create more AI." Are you sure? Lots of machine-learning/optimization processes already work much like evolution. How many people thought an email worm could bring down thousands of machines around the world in 1988? http://en.wikipedia.org/wiki/Morris_worm http://en.wikipedia.org/wiki/Morris_worm If one (or several) of the robots depicted in the below video wants someone dead, the "stair step" defense is not going to be enough: https://www.youtube.com/watch?v=QVdQM47Av20 https://www.youtube.com/watch?v=QVdQM47Av20
  - chubot 12y ago
    Well, I don't want to dismiss the possibility too quickly, since there appear to be researchers taking it seriously: http://futureoflife.org/static/data/documents/research_priorities.pdf http://futureoflife.org/static/data/documents/research_prior... (section 3.4, Control, although the general skepticism is noted) But I will say that there is a difference between feedback and unconstrained evolution. Machine learning mechanisms are feedback loops. But that doesn't mean they will break out of their own paradigm without human intervention. Malware is actually a good example. The Morris worm is a static piece of program text, invented by a human. It took advantage of a homogeneous and static environment to spread widely (lots of computers running the same program with the same vulnerability.) There's a huge difference between spreading in this environment and adapting / reproducing so as to spread in novel environments. Think about Stuxnet. AFAICT, this was a completely novel solution to a problem that had never been seen before: jumping an air gap and destroying a piece of hardware via software. What do you think is more fruitful approach to designing something like Stuxnet? 1) Assemble the best and brightest minds from multiple domains, painstakingly build simulation environments based on the most plausible intelligence, test custom malware laboriously in those environments, etc. 2) Start with some existing malware (not unlike the Morris worm), and design a meta-algorithm to evolve it into something capable of destroying a nuclear plant from afar? #2 is science fiction, as far as I can tell. The level of ingenuity required for a task like this is just qualitatively outside the domain of computers. This is closer to what would be required for us to lose control: for AI to be designing AI, rather than humans designing AI. And keep in mind that AI designing AI is harder than AI designing malware. I have seen the demos from Deep Mind where it learns to play video games without any knowledge of the rules. I don't claim to fully understand these techniques. (I have however worked in a research lab on robotics, and seen the vast gulf between what the media portrays and reality.) But I would be interested if anyone knowledgeable in those techniques sees a realistic path from deep learning to something like #2.
jeffreyrogers 12y ago
> It is the period between sentience and "advanced" artificial intelligence that should be worrying for two reasons: First, it is close at hand. More importantly, the unintended consequences of tech are never well thought out in the initial stages of adoption. Pretty much every academic that I'm aware of who works in the field of AI has similar views to Ng, i.e., suggestions of an impending AI cataclysm are wildly overblown and don't fit the facts. The people predicting doom are those who for the most part aren't involved with current research and as such don't have a good perspective on the limitations of current techniques. There is no magic in modern AI advances and there is no reason to expect that they'll yield anything like generalized intelligence. By the nature of the methods they work well in the specific domains they're trained on and poorly everywhere else.
- soup10 12y ago
  The difference between a ball of uranium and a nuclear bomb is... a slightly bigger ball. Dangerous autonomous software is very easy to imagine. What if someone hid code in flight control software that activated at a given time and sent all planes flying into the nearest building. Software is only safe because we program it to be safe. The more capable and widespread software becomes the more dangerous it becomes.
  - jsprogrammer 12y ago
    >The difference between a ball of uranium and a nuclear bomb is... a slightly bigger ball. And a very precise detonation device strapped around said ball of uranium. Software is not by itself dangerous. It only exists in the most abstract sense. What is dangerous is creating physical devices that have the ability to cause physical damage if manipulated in a certain way. Allowing those devices to be controlled autonomously through a software-hardware interface is where the danger from "AI" comes in.
    - MattHeard 12y ago
      Or if you plug the software into major financial trading hubs... oh...
billsimpson 12y ago
Accidentally creating sentient AI with today's technology would be like accidentally creating a nuclear explosion by rubbing two sticks together.
- djulius 12y ago
  Couldn't agree much. Every time there is some advance in so called "AI", killer robot and others strike back. These are just advances in doing things human can do, i.e. recognizing in pictures, driving a car. Deep learning is known since long, calculation were just too long. Fundamentally nothing has changed, we're not closer to an "intelligent machine".
  - deleted 12y ago
    [deleted]
PakG1 12y ago
What I wish I'd see more of is a fear of AI even without general intelligence. When you combine autonomy with AI, even stupid AI, you can still have bad results and significant collateral damage. See Knight Capital's trading bug, Roombas that can't distinguish between dust and dog poo, and more. Yes, Google's self-driving cars may end up being more reliable and safe than average human drivers. But that would assume the engineering team has a good handle on all situations and bugs. I'm not saying hold up progress on this front. I'm only saying that people seem to be happily buying into even trivial AI advances without both eyes open.
- collyw 12y ago
  Is this not more of a general problem where as a software engineer you need to think of ALL possible outcomes rather than just the expected ones?
karmacondon 12y ago
This appears to be an unpopular idea on hn, but: We are nowhere close to developing sentient machines. We have made no progress toward it, we're not getting closer. Statistical optimization and sentience are parallel lines. No matter how far you travel down one, you don't get any closer to the other. If "evil killer robots" are created, it won't be because of advances in big data or deep learning or any other buzzword. It'll happen independently, potentially aided by current state of the art techniques but not because of them. It could happen at any time, some guy in his basement could be on the verge of a breakthrough as we speak. But google's ability to detect cat faces or Watson's winning on jeopardy are not signs of a coming apocalypse. It's highly unlikely that we can cluster or gradient descent our way to human level of intelligence, no matter how many cores or hidden layers are used. This whole "famous people are worried about AI" thing is understandably media friendly, but it's a bit of a distraction. Fortunately the majority of active researchers aren't taking it seriously at all.
- 3pt14159 12y ago
  If nature clustered or gradient'ed its way to human intelligence why can't we? We have algorithms that can preform as well as fruit flies in simulations; why can't we continue to expand on the quality of our simulations and algorithmic approaches?
- olalonde 12y ago
  To be honest, I'm far more worried by cellular level simulations like OpenWorm[0] because there seems to be a clearer path to human level AI: more processing power and more advanced knowledge of brain biology. Given enough time and effort, it seems inevitable to me. Luckily or unluckily - depending on your perspective - we're still far from simulating a human brain. [0] http://www.artificialbrains.com/openworm http://www.artificialbrains.com/openworm
  - djulius 12y ago
    "The modelling data that is processed by the engine is written in XML." You don't need to fear much, by the time the engine evolves data into something scary, the universe is long gone.
narrator 12y ago
In Yemen or rural Afghanistan or Pakistan the inhabitants already have to deal with killer flying robots that are currently people operated but facing a manpower shortage. They will probably eventually be 100% automated. Just like any military action, whether the robots/algorithms or people who carry out these actions are perceived as evil or not depends on which country one lives in and what one's political beliefs are and whether one knows and how one feels about the people killed by said actions. In the future I expect it will be exactly the same. The future is already here, it's just not evenly distributed.
- jsprogrammer 12y ago
  Flying robots with missiles/guns/high-power lasers/weapons should never be automated.
  - gojomo 12y ago
    And this "should never" pronouncement will be enforced by you and what army?
    - jsprogrammer 12y ago
      Maybe we should have global drone coverage armed with nuclear bombs? MAD's the only way to do, right?
- notatoad 12y ago
  An autonomous drone that has been tasked to kill a human and a machine that decides on its own to kill a human are very much not the same thing. Autonomous killing is nothing new or novel - we've had sentry guns for many years now.

by rrggrr 12y ago

chubot 12y ago

I've been thinking about what I visualize as "plant" intelligence, in response to Elon Musk's statements about killer AI. It seems like Andrew Ng is saying the same thing. The metaphor is that you could have "sedentary" superintelligences that solve a relatively narrow problem. They take in huge amounts of data, and spit out ingenious answers to well-defined problems. Things like traffic-routing, energy allocation, or photo/video recognition, etc. They are tools for humans. By what mechanism do these superintelligences involve into organisms motivated to extinguish humans? Reproduction, which leads to independent evolution. That process is inherently unpredictable. But right now we're firmly in the stage of humans creating AI. Our AI is nowhere near capable enough to create more AI. And I would argue that there is no possibility of this happening by "accident"; you would have to specifically engineer AI to able to create more AI. I agree with Ng in that it's more of a distraction now than anything else. The jobs issue is a lot more pertinent. It reminds me of this book I got at Google 8 or 9 years ago. This guy asked a robotic professors how they would avoid being killed by a robot. And the professor said: "Climb up one step" (on a staircase).

gojomo 12y ago
"Our AI is nowhere near capable enough to create more AI." Are you sure? Lots of machine-learning/optimization processes already work much like evolution. How many people thought an email worm could bring down thousands of machines around the world in 1988? http://en.wikipedia.org/wiki/Morris_worm http://en.wikipedia.org/wiki/Morris_worm If one (or several) of the robots depicted in the below video wants someone dead, the "stair step" defense is not going to be enough: https://www.youtube.com/watch?v=QVdQM47Av20 https://www.youtube.com/watch?v=QVdQM47Av20
- chubot 12y ago
  Well, I don't want to dismiss the possibility too quickly, since there appear to be researchers taking it seriously: http://futureoflife.org/static/data/documents/research_priorities.pdf http://futureoflife.org/static/data/documents/research_prior... (section 3.4, Control, although the general skepticism is noted) But I will say that there is a difference between feedback and unconstrained evolution. Machine learning mechanisms are feedback loops. But that doesn't mean they will break out of their own paradigm without human intervention. Malware is actually a good example. The Morris worm is a static piece of program text, invented by a human. It took advantage of a homogeneous and static environment to spread widely (lots of computers running the same program with the same vulnerability.) There's a huge difference between spreading in this environment and adapting / reproducing so as to spread in novel environments. Think about Stuxnet. AFAICT, this was a completely novel solution to a problem that had never been seen before: jumping an air gap and destroying a piece of hardware via software. What do you think is more fruitful approach to designing something like Stuxnet? 1) Assemble the best and brightest minds from multiple domains, painstakingly build simulation environments based on the most plausible intelligence, test custom malware laboriously in those environments, etc. 2) Start with some existing malware (not unlike the Morris worm), and design a meta-algorithm to evolve it into something capable of destroying a nuclear plant from afar? #2 is science fiction, as far as I can tell. The level of ingenuity required for a task like this is just qualitatively outside the domain of computers. This is closer to what would be required for us to lose control: for AI to be designing AI, rather than humans designing AI. And keep in mind that AI designing AI is harder than AI designing malware. I have seen the demos from Deep Mind where it learns to play video games without any knowledge of the rules. I don't claim to fully understand these techniques. (I have however worked in a research lab on robotics, and seen the vast gulf between what the media portrays and reality.) But I would be interested if anyone knowledgeable in those techniques sees a realistic path from deep learning to something like #2.

jeffreyrogers 12y ago

> It is the period between sentience and "advanced" artificial intelligence that should be worrying for two reasons: First, it is close at hand. More importantly, the unintended consequences of tech are never well thought out in the initial stages of adoption. Pretty much every academic that I'm aware of who works in the field of AI has similar views to Ng, i.e., suggestions of an impending AI cataclysm are wildly overblown and don't fit the facts. The people predicting doom are those who for the most part aren't involved with current research and as such don't have a good perspective on the limitations of current techniques. There is no magic in modern AI advances and there is no reason to expect that they'll yield anything like generalized intelligence. By the nature of the methods they work well in the specific domains they're trained on and poorly everywhere else.

soup10 12y ago
The difference between a ball of uranium and a nuclear bomb is... a slightly bigger ball. Dangerous autonomous software is very easy to imagine. What if someone hid code in flight control software that activated at a given time and sent all planes flying into the nearest building. Software is only safe because we program it to be safe. The more capable and widespread software becomes the more dangerous it becomes.
- jsprogrammer 12y ago
  >The difference between a ball of uranium and a nuclear bomb is... a slightly bigger ball. And a very precise detonation device strapped around said ball of uranium. Software is not by itself dangerous. It only exists in the most abstract sense. What is dangerous is creating physical devices that have the ability to cause physical damage if manipulated in a certain way. Allowing those devices to be controlled autonomously through a software-hardware interface is where the danger from "AI" comes in.
  - MattHeard 12y ago
    Or if you plug the software into major financial trading hubs... oh...

billsimpson 12y ago

Accidentally creating sentient AI with today's technology would be like accidentally creating a nuclear explosion by rubbing two sticks together.

djulius 12y ago
Couldn't agree much. Every time there is some advance in so called "AI", killer robot and others strike back. These are just advances in doing things human can do, i.e. recognizing in pictures, driving a car. Deep learning is known since long, calculation were just too long. Fundamentally nothing has changed, we're not closer to an "intelligent machine".
- deleted 12y ago
  [deleted]

PakG1 12y ago

What I wish I'd see more of is a fear of AI even without general intelligence. When you combine autonomy with AI, even stupid AI, you can still have bad results and significant collateral damage. See Knight Capital's trading bug, Roombas that can't distinguish between dust and dog poo, and more. Yes, Google's self-driving cars may end up being more reliable and safe than average human drivers. But that would assume the engineering team has a good handle on all situations and bugs. I'm not saying hold up progress on this front. I'm only saying that people seem to be happily buying into even trivial AI advances without both eyes open.

collyw 12y ago
Is this not more of a general problem where as a software engineer you need to think of ALL possible outcomes rather than just the expected ones?

karmacondon 12y ago

This appears to be an unpopular idea on hn, but: We are nowhere close to developing sentient machines. We have made no progress toward it, we're not getting closer. Statistical optimization and sentience are parallel lines. No matter how far you travel down one, you don't get any closer to the other. If "evil killer robots" are created, it won't be because of advances in big data or deep learning or any other buzzword. It'll happen independently, potentially aided by current state of the art techniques but not because of them. It could happen at any time, some guy in his basement could be on the verge of a breakthrough as we speak. But google's ability to detect cat faces or Watson's winning on jeopardy are not signs of a coming apocalypse. It's highly unlikely that we can cluster or gradient descent our way to human level of intelligence, no matter how many cores or hidden layers are used. This whole "famous people are worried about AI" thing is understandably media friendly, but it's a bit of a distraction. Fortunately the majority of active researchers aren't taking it seriously at all.

3pt14159 12y ago
If nature clustered or gradient'ed its way to human intelligence why can't we? We have algorithms that can preform as well as fruit flies in simulations; why can't we continue to expand on the quality of our simulations and algorithmic approaches?
olalonde 12y ago
To be honest, I'm far more worried by cellular level simulations like OpenWorm[0] because there seems to be a clearer path to human level AI: more processing power and more advanced knowledge of brain biology. Given enough time and effort, it seems inevitable to me. Luckily or unluckily - depending on your perspective - we're still far from simulating a human brain. [0] http://www.artificialbrains.com/openworm http://www.artificialbrains.com/openworm
- djulius 12y ago
  "The modelling data that is processed by the engine is written in XML." You don't need to fear much, by the time the engine evolves data into something scary, the universe is long gone.

narrator 12y ago

In Yemen or rural Afghanistan or Pakistan the inhabitants already have to deal with killer flying robots that are currently people operated but facing a manpower shortage. They will probably eventually be 100% automated. Just like any military action, whether the robots/algorithms or people who carry out these actions are perceived as evil or not depends on which country one lives in and what one's political beliefs are and whether one knows and how one feels about the people killed by said actions. In the future I expect it will be exactly the same. The future is already here, it's just not evenly distributed.

jsprogrammer 12y ago
Flying robots with missiles/guns/high-power lasers/weapons should never be automated.
- gojomo 12y ago
  And this "should never" pronouncement will be enforced by you and what army?
  - jsprogrammer 12y ago
    Maybe we should have global drone coverage armed with nuclear bombs? MAD's the only way to do, right?
notatoad 12y ago
An autonomous drone that has been tasked to kill a human and a machine that decides on its own to kill a human are very much not the same thing. Autonomous killing is nothing new or novel - we've had sentry guns for many years now.