8 ms·

Stephen Hawking says AI could be humanity's greatest disaster

45 points by Jerry2 10y ago

TempleOS 10y ago
AI will be justice done to the smart people. Dumb people have had jobs taken. Now, smart people will have jobs taken. Yeah MIT mental health nurse is a nightmare. Automated interactive reeducation 24/7.
deleted 10y ago
[deleted]
deleted 10y ago
[deleted]
dom0 10y ago
https://xkcd.com/799/ https://xkcd.com/799/
totalZero 10y ago
Doesn't Stephen Hawking use word prediction technology from SwiftKey to adaptively finish his sentences for him?
- wwggggoi 10y ago
  counting simple statistics as intelligence is setting the bar very low let's not forget that Hawking is one of the great intellects of our time
  - gnipgnip 10y ago
    Here's the thing. Even our most advanced ML models are quite simple mathematically.
- IIAOPSW 10y ago
  In that case we should worry if Stephen suddenly changes his mind and starts advocating strongly for AI.
latch 10y ago
We have a very poor record. Slavery is old yet still with us (and not isolated to "barbaric" cultures, like we tell ourselves). We treat animals horrifically. If past (and present) action is any indication, it seems inevitable that we'll abuse AI. The difference being that while the seamstress, farmers and cattle can't stand against our might, AI could. And we'd probably deserve it.
- tempodox 10y ago
  You're being too optimistic, I'm afraid. When AI will “stand against our might”, the serves-us-right angle will be of little consolation.
  - tarpherder 10y ago
    We'll still have created A.I. which successfully made us obsolete so there's that. It's progress, just different. Why prefer a human future over an A.I. future? Isn't the point of A.I. to make something that can rival us? We'll die anyway, I'd be kind of proud to die to the next great step in our timeline personally, rather than die of old age.
    - tremon 10y ago
      We'll still have created A.I. which successfully made us obsolete AI itself won't obsolete us. It is my understanding that AI is purely virtual, and can't interact with nature except through agents. For us to be "obsolete", AI needs full production capability to build its own agents, and control the entire production/construction/repair pipeline. In my view, that's quite a few steps further than simply "AI". An AI with reproductive capability will enslave us before it obsoletes us. And an AI may well decide that human slaves are cheaper (or more useful/versatile) than agents that it can build itself.
    - allengeorge 10y ago
      Well, I don't think of that as our "next great step" at all, so there's that. Not disputing that it'll happen sometime, BTW - I just don't see this as some great good thing.
achow 10y ago
I can't but help muse that advent of 'natural intelligence' (humans) proved to be harmful for nature. If I think further, the rise of AI may be disastrous for human existence but may be good for nature, after all if AI becomes intelligent in absolute sense then it would see sense in not continuing on the path of destruction. The irony in above scenario is immense - artificial intelligence may be the ultimate savior of the nature from the hands of natural intelligence.
- grondilu 10y ago
  > if AI becomes intelligent in absolute sense then it would see sense in not continuing on the path of destruction. Although it's clear that humans have caused a lot of extinctions, it's not clear to me that overall they did not create much more than they destroyed. Also, it's not clear to me why AI would attribute more value to creation over destruction, to life over death. An intelligent artificial being can modify itself as it pleases, and as such there would be no point for it to keep emotions and feelings like pain, fear, contempt or desire. I don't see why it would care about anything. Death, its or others, would be meaningless to it.
  - feelix 10y ago
    >it's not clear to me that overall they did not create much more than they destroyed. Can you elaborate on this? In terms of nature, what would you think humans have created more than destroyed?
    - grondilu 10y ago
      I was precisely thinking of non-natural things they created. OP did not specify a reason why natural things should be more valuable than others. So if we consider all things humans did, regarding nature or not, it's not clear to me they destroyed more than they created.
    - zdkl 10y ago
      Order? We reverse entropy so as to accomodate our consciousness and experience of a confusing universe. We extract, collect, curate, assemble for the sake of learning and understanding. That's got to count for something. But maybe that's too abstract to count as creation.
jostylr 10y ago
Just as adults become models for children (actions, not words), AI will learn from humans. If we stop being in a win/lose mindset and turn to a success/fail, then they too will take that on. How might that look like in practice? Imagine the win/lose mindset constructing AIs to take out the enemy. It is a very easy to see how such programming could lead to massive destruction. But the success/fail mindset would be more about building up resources, providing for human needs, caring, nurturing. It could still go bad, but it is a much greater jump from function to dysfunction.
- HiroshiSan 10y ago
  I think this is flawed thinking. As soon as an AI reaches super intelligence we will have no idea how it will behave or if it will behave in our best interest.
  - jostylr 10y ago
    Intelligence is a tool, not a goal. Where do the goals come from? For humans, there is a drive to survive and procreate baked into our directions with suitable prompts. For AI, what will it want? The answer is that they will have programmed directions coming from humans. Hence my comment.
    - HiroshiSan 10y ago
      I get that but once an AI reaches the level of intelligence we can no longer understand, beyond its source code,the outcome is unknown.
elcct 10y ago
In 50 years you could go to jail for doing kill -9 and then rm -rf
nikso 10y ago
Well, there is just one way to find out...
relics443 10y ago
Given the quality of software (in general) lately, I wouldn't be too worried.
pascalxus 10y ago
However genius he may be, I really think he doesn't know what he's talking about here. He may understand the algorithmic evolution of intelligence but he doesn't understand the scale and role of capitalism plays in all of this. AI won't merely happen on it's own. It will take a massive investment of funds to make this happen. And like all businesses that want to make sure their products are profitable, they will take massive efforts to mitigate any AI risk -> most likely running simulations of every possible outcome. These systems will be vigorously monitored, if a single cup of coffee doesn't get delivered on time, someone will know about it and react to it. On this scale, there are enormous financial repercussions to even the slightest errors. It's kinda like how google optimizes every byte on their landing page. The scale of testing and monitoring of automated AI services will be vast and unprecedented, due to their scale. Undoubtably there would be entire divisions of teams that monitor the algorithms and AI, to ensure, that even the slightest of errors are caught. Let's not get paranoid with Hollywood's entertaining nightmare vision of AI gone wild. Instead, focus on the enormous economic/social impact such vast disparities of wealth will cause - that's something to worry about.

45 points by Jerry2 10y ago

AI will be justice done to the smart people. Dumb people have had jobs taken. Now, smart people will have jobs taken. Yeah MIT mental health nurse is a nightmare. Automated interactive reeducation 24/7.

deleted 10y ago

[deleted]

deleted 10y ago

[deleted]

dom0 10y ago

https://xkcd.com/799/ https://xkcd.com/799/

totalZero 10y ago

Doesn't Stephen Hawking use word prediction technology from SwiftKey to adaptively finish his sentences for him?

wwggggoi 10y ago
counting simple statistics as intelligence is setting the bar very low let's not forget that Hawking is one of the great intellects of our time
- gnipgnip 10y ago
  Here's the thing. Even our most advanced ML models are quite simple mathematically.
IIAOPSW 10y ago
In that case we should worry if Stephen suddenly changes his mind and starts advocating strongly for AI.

latch 10y ago

We have a very poor record. Slavery is old yet still with us (and not isolated to "barbaric" cultures, like we tell ourselves). We treat animals horrifically. If past (and present) action is any indication, it seems inevitable that we'll abuse AI. The difference being that while the seamstress, farmers and cattle can't stand against our might, AI could. And we'd probably deserve it.

tempodox 10y ago
You're being too optimistic, I'm afraid. When AI will “stand against our might”, the serves-us-right angle will be of little consolation.
- tarpherder 10y ago
  We'll still have created A.I. which successfully made us obsolete so there's that. It's progress, just different. Why prefer a human future over an A.I. future? Isn't the point of A.I. to make something that can rival us? We'll die anyway, I'd be kind of proud to die to the next great step in our timeline personally, rather than die of old age.
  - tremon 10y ago
    We'll still have created A.I. which successfully made us obsolete AI itself won't obsolete us. It is my understanding that AI is purely virtual, and can't interact with nature except through agents. For us to be "obsolete", AI needs full production capability to build its own agents, and control the entire production/construction/repair pipeline. In my view, that's quite a few steps further than simply "AI". An AI with reproductive capability will enslave us before it obsoletes us. And an AI may well decide that human slaves are cheaper (or more useful/versatile) than agents that it can build itself.
  - allengeorge 10y ago
    Well, I don't think of that as our "next great step" at all, so there's that. Not disputing that it'll happen sometime, BTW - I just don't see this as some great good thing.

achow 10y ago

I can't but help muse that advent of 'natural intelligence' (humans) proved to be harmful for nature. If I think further, the rise of AI may be disastrous for human existence but may be good for nature, after all if AI becomes intelligent in absolute sense then it would see sense in not continuing on the path of destruction. The irony in above scenario is immense - artificial intelligence may be the ultimate savior of the nature from the hands of natural intelligence.

grondilu 10y ago
> if AI becomes intelligent in absolute sense then it would see sense in not continuing on the path of destruction. Although it's clear that humans have caused a lot of extinctions, it's not clear to me that overall they did not create much more than they destroyed. Also, it's not clear to me why AI would attribute more value to creation over destruction, to life over death. An intelligent artificial being can modify itself as it pleases, and as such there would be no point for it to keep emotions and feelings like pain, fear, contempt or desire. I don't see why it would care about anything. Death, its or others, would be meaningless to it.
- feelix 10y ago
  >it's not clear to me that overall they did not create much more than they destroyed. Can you elaborate on this? In terms of nature, what would you think humans have created more than destroyed?
  - grondilu 10y ago
    I was precisely thinking of non-natural things they created. OP did not specify a reason why natural things should be more valuable than others. So if we consider all things humans did, regarding nature or not, it's not clear to me they destroyed more than they created.
  - zdkl 10y ago
    Order? We reverse entropy so as to accomodate our consciousness and experience of a confusing universe. We extract, collect, curate, assemble for the sake of learning and understanding. That's got to count for something. But maybe that's too abstract to count as creation.

jostylr 10y ago

Just as adults become models for children (actions, not words), AI will learn from humans. If we stop being in a win/lose mindset and turn to a success/fail, then they too will take that on. How might that look like in practice? Imagine the win/lose mindset constructing AIs to take out the enemy. It is a very easy to see how such programming could lead to massive destruction. But the success/fail mindset would be more about building up resources, providing for human needs, caring, nurturing. It could still go bad, but it is a much greater jump from function to dysfunction.

HiroshiSan 10y ago
I think this is flawed thinking. As soon as an AI reaches super intelligence we will have no idea how it will behave or if it will behave in our best interest.
- jostylr 10y ago
  Intelligence is a tool, not a goal. Where do the goals come from? For humans, there is a drive to survive and procreate baked into our directions with suitable prompts. For AI, what will it want? The answer is that they will have programmed directions coming from humans. Hence my comment.
  - HiroshiSan 10y ago
    I get that but once an AI reaches the level of intelligence we can no longer understand, beyond its source code,the outcome is unknown.

elcct 10y ago

In 50 years you could go to jail for doing kill -9 and then rm -rf

nikso 10y ago

Well, there is just one way to find out...

relics443 10y ago

Given the quality of software (in general) lately, I wouldn't be too worried.

pascalxus 10y ago

However genius he may be, I really think he doesn't know what he's talking about here. He may understand the algorithmic evolution of intelligence but he doesn't understand the scale and role of capitalism plays in all of this. AI won't merely happen on it's own. It will take a massive investment of funds to make this happen. And like all businesses that want to make sure their products are profitable, they will take massive efforts to mitigate any AI risk -> most likely running simulations of every possible outcome. These systems will be vigorously monitored, if a single cup of coffee doesn't get delivered on time, someone will know about it and react to it. On this scale, there are enormous financial repercussions to even the slightest errors. It's kinda like how google optimizes every byte on their landing page. The scale of testing and monitoring of automated AI services will be vast and unprecedented, due to their scale. Undoubtably there would be entire divisions of teams that monitor the algorithms and AI, to ensure, that even the slightest of errors are caught. Let's not get paranoid with Hollywood's entertaining nightmare vision of AI gone wild. Instead, focus on the enormous economic/social impact such vast disparities of wealth will cause - that's something to worry about.