3 ms·

It's odd that all of the very smart people of the world feel AI is a threat to humanity. Couldn't we just like, not allow it to _be_ a threat in the first plac

It's odd that all of the very smart people of the world feel AI is a threat to humanity. Couldn't we just like, not allow it to _be_ a threat in the first place? One would think that morality gates would be one of the first hurdles in creating AI.

altcognito 12y ago
One would think morality gates would be one of the first hurdles in all human endeavors, yet here we are.
- Turing_Machine 12y ago
  Indeed. Even agreeing on a universal definition for "morality" seems to have eluded us.
davidrusu 12y ago
There are smart people working on it, see miri: https://intelligence.org/ https://intelligence.org/ But it's a very hard problem. To get a better feel for the problem I suggest you read Superintelligence by Nick Bostrom, it's what convinced Elon Musk of the dangers of AI: https://twitter.com/elonmusk/status/495759307346952192 https://twitter.com/elonmusk/status/495759307346952192
deepsun 12y ago
Selection bias. Those who don't think AI a threat don't appear in the news. News audience pays for scary stories.
- davidrusu 12y ago
  Ironically, from the head of Microsoft Research: http://www.smh.com.au/digital-life/digital-life-news/artificial-intelligence-not-a-threat-microsofts-eric-horvitz-contradicts-elon-musk-stephen-hawking-bill-gates-20150129-130sgv.html http://www.smh.com.au/digital-life/digital-life-news/artific...
monk_e_boy 12y ago
It lives in a computer. Can't we just turn it off?
- Turing_Machine 12y ago
  If it truly has human-level (or greater) intelligence, you're going to have some ethical issues that crop up with turning it off. Even if you're personally okay with it, there are going to be other people who aren't. Waiting until it commits a crime and then having some kind of "trial" and "execution" may not be an option. If it's smart/powerful enough, there may not be enough time for that.
  - vinceguidry 12y ago
    That's not the problem. The problem is that when we are reliant on intelligent systems to, say, detect certain kinds of fraud in financial markets, then turning them off would have huge deleterious effects all on their own.
    - Turing_Machine 12y ago
      It's not a problem now. If we get machines anywhere near as smart as people, the AI rights organizations will soon follow.
      - vinceguidry 12y ago
        You mean like the people trying to give plants rights? Sure, they might spring up, doesn't mean they'll get anywhere. For better or worse, altruism will forever follow economics. The civil war didn't end the Atlantic slave trade, technology eventually made it largely uneconomical, allowing those wanting to end slavery the political ability to outlaw it. Even that didn't stop the slave trade. It took extra-special attention from the British Navy to actually enforce a slave trade ban on the open seas. 150 years later and black people are still getting the short end of the stick. By the time AI gets real rights, it will be long past the time where most of us believe they should get them.
wslh 12y ago
The truth is... nobody can predict the future, so what Bill Gates and any other person thinks is a matter of speculation. Personally I think it is a threat!
stephenboyd 12y ago
If by "morality gates" you're talking about about an ethical filter that all intelligent machine actions have to go through, this can't be "one of the first hurdles" because such a filter would need to be a powerful AI application itself. And there's no guarantee that self-imposed limitations will always be applied. Someone could just rewrite and recompile without the filters turned on. Imagine if the US could have built in "morality gates" to the first nuclear weapons. That wouldn't have stopped other countries from building the weapons without the restrictions. And we know software is infinitely configurable and modifiable if you have the source code.
deleted 12y ago
[deleted]
skj 12y ago
> It's odd that all of the very smart people of the world feel AI is a threat to humanity. Only if you define one of the conditions for being a "very smart person" to be feeling that AI is a threat to humanity.
PeterisP 12y ago
The short answer is that: (a) we don't really know how to define a morality for a self-improving optimization system to allow a potential artificial general intelligence (AGI) not to be a threat in the first place; (b) if an entity arrives (no matter if it's an AGI, some aliens or Jesus) that is more powerful than us, then if its goals & morality aren't aligned with ours, then we're screwed. And almost all of the space of possible goals either doesn't include humanity at all or includes humanity as a glorified zoo exhibit - only a very, very narrow and specific set of optimization goals (that we're currently unable to formally describe) would mean a happy future for us. If you think that it's easy, think twice - all the obvious naive goals of 'make us happy' or 'keep us safe' or 'do what I want' actually turn into horrible dystopias as soon as you try to formally define them as instructions for someone/something to maximize that goal. (c) functioning morality is not a requirement for power - if somehow we solve the very hard problem of how to create an AGI before we solve the very hard problem on how to make such an AGI be friendly to us, then we lose. Permanently.
drawkbox 12y ago
The smart/wealthy are also powerful and rogue agents will definitely use AI to do things. AI that wealth can't necessarily control will definitely exist and that is probably system changing. I doubt it will be a threat to humanity anytime soon like a Grey Goo type situation, but even in such things at HFT the influence is immense. People will definitely use it to do power plays on others, we shouldn't have dropped nukes but we have done plenty of that. Basically the warning in 2001 was that AI was not to fear really but the subversive programming by other humans within the AI systems. I am sure over time AI systems that access critical infrastructure and financial systems will have to operate in a distributed approval/review system checked by other agents and maybe even a blockchain like system.
deleted 12y ago
[deleted]

altcognito 12y ago

One would think morality gates would be one of the first hurdles in all human endeavors, yet here we are.

Turing_Machine 12y ago
Indeed. Even agreeing on a universal definition for "morality" seems to have eluded us.

davidrusu 12y ago

There are smart people working on it, see miri: https://intelligence.org/ https://intelligence.org/ But it's a very hard problem. To get a better feel for the problem I suggest you read Superintelligence by Nick Bostrom, it's what convinced Elon Musk of the dangers of AI: https://twitter.com/elonmusk/status/495759307346952192 https://twitter.com/elonmusk/status/495759307346952192

deepsun 12y ago

Selection bias. Those who don't think AI a threat don't appear in the news. News audience pays for scary stories.

davidrusu 12y ago
Ironically, from the head of Microsoft Research: http://www.smh.com.au/digital-life/digital-life-news/artificial-intelligence-not-a-threat-microsofts-eric-horvitz-contradicts-elon-musk-stephen-hawking-bill-gates-20150129-130sgv.html http://www.smh.com.au/digital-life/digital-life-news/artific...

monk_e_boy 12y ago

It lives in a computer. Can't we just turn it off?

Turing_Machine 12y ago
If it truly has human-level (or greater) intelligence, you're going to have some ethical issues that crop up with turning it off. Even if you're personally okay with it, there are going to be other people who aren't. Waiting until it commits a crime and then having some kind of "trial" and "execution" may not be an option. If it's smart/powerful enough, there may not be enough time for that.
- vinceguidry 12y ago
  That's not the problem. The problem is that when we are reliant on intelligent systems to, say, detect certain kinds of fraud in financial markets, then turning them off would have huge deleterious effects all on their own.
  - Turing_Machine 12y ago
    It's not a problem now. If we get machines anywhere near as smart as people, the AI rights organizations will soon follow.
    - vinceguidry 12y ago
      You mean like the people trying to give plants rights? Sure, they might spring up, doesn't mean they'll get anywhere. For better or worse, altruism will forever follow economics. The civil war didn't end the Atlantic slave trade, technology eventually made it largely uneconomical, allowing those wanting to end slavery the political ability to outlaw it. Even that didn't stop the slave trade. It took extra-special attention from the British Navy to actually enforce a slave trade ban on the open seas. 150 years later and black people are still getting the short end of the stick. By the time AI gets real rights, it will be long past the time where most of us believe they should get them.

wslh 12y ago

The truth is... nobody can predict the future, so what Bill Gates and any other person thinks is a matter of speculation. Personally I think it is a threat!

stephenboyd 12y ago

If by "morality gates" you're talking about about an ethical filter that all intelligent machine actions have to go through, this can't be "one of the first hurdles" because such a filter would need to be a powerful AI application itself. And there's no guarantee that self-imposed limitations will always be applied. Someone could just rewrite and recompile without the filters turned on. Imagine if the US could have built in "morality gates" to the first nuclear weapons. That wouldn't have stopped other countries from building the weapons without the restrictions. And we know software is infinitely configurable and modifiable if you have the source code.

deleted 12y ago

[deleted]

skj 12y ago

> It's odd that all of the very smart people of the world feel AI is a threat to humanity. Only if you define one of the conditions for being a "very smart person" to be feeling that AI is a threat to humanity.

PeterisP 12y ago

The short answer is that: (a) we don't really know how to define a morality for a self-improving optimization system to allow a potential artificial general intelligence (AGI) not to be a threat in the first place; (b) if an entity arrives (no matter if it's an AGI, some aliens or Jesus) that is more powerful than us, then if its goals & morality aren't aligned with ours, then we're screwed. And almost all of the space of possible goals either doesn't include humanity at all or includes humanity as a glorified zoo exhibit - only a very, very narrow and specific set of optimization goals (that we're currently unable to formally describe) would mean a happy future for us. If you think that it's easy, think twice - all the obvious naive goals of 'make us happy' or 'keep us safe' or 'do what I want' actually turn into horrible dystopias as soon as you try to formally define them as instructions for someone/something to maximize that goal. (c) functioning morality is not a requirement for power - if somehow we solve the very hard problem of how to create an AGI before we solve the very hard problem on how to make such an AGI be friendly to us, then we lose. Permanently.

drawkbox 12y ago

The smart/wealthy are also powerful and rogue agents will definitely use AI to do things. AI that wealth can't necessarily control will definitely exist and that is probably system changing. I doubt it will be a threat to humanity anytime soon like a Grey Goo type situation, but even in such things at HFT the influence is immense. People will definitely use it to do power plays on others, we shouldn't have dropped nukes but we have done plenty of that. Basically the warning in 2001 was that AI was not to fear really but the subversive programming by other humans within the AI systems. I am sure over time AI systems that access critical infrastructure and financial systems will have to operate in a distributed approval/review system checked by other agents and maybe even a blockchain like system.

deleted 12y ago

[deleted]