5 ms·

The letter isn't (just) about modern weapon systems. It was put together by this group: http://futureoflife.org/ai-news/ http://futureoflife.org/ai-news/ Also,

The letter isn't (just) about modern weapon systems. It was put together by this group: http://futureoflife.org/ai-news/ http://futureoflife.org/ai-news/ Also, no-one is worried about a skynet scenario. The worrying scenario is just any powerful system that optimises for something that's different from what humans want. Second, the point is that even uncertainty is enough for action. For AI to not be a problem, you'd need to be very confident that it'll occur a long way in the future, and that there's nothing we can do until it's closer. As you've said, we don't have confidence in the timeline. We have large uncertainty. And that's more reason for action, especially research. Consider analogously: "We've got no idea what the chance of run-away climate change is, so we shouldn't do anything about it." Seems like a bad argument to me.

andreyf 11y ago
Also, no-one is worried about a skynet scenario. The worrying scenario is just any powerful system that optimises for something that's different from what humans want. So Transcendence in leu of The Terminator. It's still hollywood fiction.
- Moshe_Silnorin 11y ago
  Global warming will not happen because The Day After Tomorrow is a movie. 12 Monkeys is a movie therefore weaponized biotechnology is without risk. "Hollywood made a movie vaguely like this" isn't much of an argument against anything.
  - andreyf 11y ago
    Don't straw man me, bro. My point is that the entire idea of an buggy AI destroying humanity is absolutely divorced from the reality of AI research. It's not unimaginable to the lay-person, but there is simply no way of getting there from the neural networks we're making now. To begin, can you go ahead and explain what self-awareness means in terms of a neural network you are training?
- deleted 11y ago
  [deleted]
manish_gill 11y ago
Except, in the case of climate research, we have a plethora of evidence of the possible harmful effects and we can see it happening today. Everything about the potential harmful effects of AI is pure conjecture, because there is no human-level general intelligence AI system that exists today. It's, once again, something philosophers like Bostrom will make a career out of. I'm 100% with Torvalds on this when he laughs at the prepostorous notion that AI will become a doomsday scenario. I think it'll become more and more specialised, branch out to other fields and become reasonably good. But there's a huge leap to go from there to HGI. > any powerful system that optimises for something that's different from what humans want. Except this notion rests on the premise that humans will not be in full control, which leads to the exponential growth argument which leads back to the Skynet like scenario. > the point is that even uncertainty is enough for action And that action is...what exactly? People won't stop building intelligent systems. There is no real path that we have from where we are to HGI, so it's not like researchers have a concrete path. Just what exactly does this research look like? > "We've got no idea what the chance of run-away climate change is, so we shouldn't do anything about it." Extremely Poor analogy. We have decades worth of concrete data that tells us the nature and reality of climate change. We demonstrably know that it's a threat. Can you say the same about AI? Also, I'll take the time to re-iterate how heavily skeptical I remain of groups like MIRI that are spearheaded by people who don't believe in the scientific method, believe in stuff like cryogenics, have history of trying to profit off of someone else's copyrighted material and have someone managed to convince a whole lot of people that donating them is the best way to fight off the AI doomsday scenario. People should do their research before linking to stuff like that. :(
- BenjaminTodd 11y ago
  I'm not saying we don't know whether climate change poses a tail risk or not (it obviously does). I'm just saying that claiming uncertainty isn't a good reason to avoid action. In general, if there's a poorly understood but potentially very bad risk, then (a) more research to understand the risk is really high priority (b) if that research doesn't rule out the really bad scenario, we should try to do something to prevent it. With AI, unfortunately waiting until the evidence that it's harmful is well established is not possible, because then it could be too late. What AI risk research could involve is laid out in detail in the link.
mxfh 11y ago
The worrying scenario is just any powerful system that optimizes for something that's different from what humans want. Like AI based-HFT? Which humans want what? I'm pretty sure, that if there's one constant, that there always will be some humans on either side of the argument.

andreyf 11y ago

Also, no-one is worried about a skynet scenario. The worrying scenario is just any powerful system that optimises for something that's different from what humans want. So Transcendence in leu of The Terminator. It's still hollywood fiction.

Moshe_Silnorin 11y ago
Global warming will not happen because The Day After Tomorrow is a movie. 12 Monkeys is a movie therefore weaponized biotechnology is without risk. "Hollywood made a movie vaguely like this" isn't much of an argument against anything.
- andreyf 11y ago
  Don't straw man me, bro. My point is that the entire idea of an buggy AI destroying humanity is absolutely divorced from the reality of AI research. It's not unimaginable to the lay-person, but there is simply no way of getting there from the neural networks we're making now. To begin, can you go ahead and explain what self-awareness means in terms of a neural network you are training?
deleted 11y ago
[deleted]

manish_gill 11y ago

Except, in the case of climate research, we have a plethora of evidence of the possible harmful effects and we can see it happening today. Everything about the potential harmful effects of AI is pure conjecture, because there is no human-level general intelligence AI system that exists today. It's, once again, something philosophers like Bostrom will make a career out of. I'm 100% with Torvalds on this when he laughs at the prepostorous notion that AI will become a doomsday scenario. I think it'll become more and more specialised, branch out to other fields and become reasonably good. But there's a huge leap to go from there to HGI. > any powerful system that optimises for something that's different from what humans want. Except this notion rests on the premise that humans will not be in full control, which leads to the exponential growth argument which leads back to the Skynet like scenario. > the point is that even uncertainty is enough for action And that action is...what exactly? People won't stop building intelligent systems. There is no real path that we have from where we are to HGI, so it's not like researchers have a concrete path. Just what exactly does this research look like? > "We've got no idea what the chance of run-away climate change is, so we shouldn't do anything about it." Extremely Poor analogy. We have decades worth of concrete data that tells us the nature and reality of climate change. We demonstrably know that it's a threat. Can you say the same about AI? Also, I'll take the time to re-iterate how heavily skeptical I remain of groups like MIRI that are spearheaded by people who don't believe in the scientific method, believe in stuff like cryogenics, have history of trying to profit off of someone else's copyrighted material and have someone managed to convince a whole lot of people that donating them is the best way to fight off the AI doomsday scenario. People should do their research before linking to stuff like that. :(

BenjaminTodd 11y ago
I'm not saying we don't know whether climate change poses a tail risk or not (it obviously does). I'm just saying that claiming uncertainty isn't a good reason to avoid action. In general, if there's a poorly understood but potentially very bad risk, then (a) more research to understand the risk is really high priority (b) if that research doesn't rule out the really bad scenario, we should try to do something to prevent it. With AI, unfortunately waiting until the evidence that it's harmful is well established is not possible, because then it could be too late. What AI risk research could involve is laid out in detail in the link.

mxfh 11y ago

The worrying scenario is just any powerful system that optimizes for something that's different from what humans want. Like AI based-HFT? Which humans want what? I'm pretty sure, that if there's one constant, that there always will be some humans on either side of the argument.