3 ms·

> Wouldn't you even agree that there is a 1% chance that AGI is possible and that it could harm us once invented? There's a small chance aliens could find us,

> Wouldn't you even agree that there is a 1% chance that AGI is possible and that it could harm us once invented? There's a small chance aliens could find us, too. Yet, we don't know any more about that than we do about AI. The most imminent threat to humans is humans. In the nearer future, it seems more likely that someone will use machine learning technology in weaponry to cause harm than create AGI. Or, that humans will undergo a global "cultural revolution" like in China or Cambodia (or ISIS now), killing anyone with an education, under the guise that "machines are evil", yet with the true goal of obtaining power, as humans often do. Ultimately, people are free to work on what they want. If some people want to work on AGI safety, that's great. Perhaps their goal is to make humanity a little more kind. That's certainly worthy. For me, I'm more interested in seeing research focused on giving better tools to doctors for diagnoses. Radiology is a great place to apply image recognition systems. We just need some good labeled datasets. Sometimes I get frustrated by the "AGI is an existential risk" quotes like Musk's because I feel that's a distraction from some really beneficial applications of machine learning that will help save lives. It could cause over-hype, leading to another AI winter, pushing back life-saving advances 15 years, as happened after the 80s. Personally I don't have a problem with the kinds of things some AGI safety researchers, like Stuart Russel, says about AGI. He isn't saying it will arrive in 2030-2040, like Musk. Musk has an incentive to capitalize on people's fear of AGI. It attracts investment in his non-profit, which can in turn help him continue building his in-house AI program at Tesla. Primarily I disagree with his conclusions about AI tech, but I'm also weary of his motivations. I think some people view him as altruistic and I just don't see that. > What would be the minimum level sufficient to convince you that AGI is possible? A system that can determine its own goals. We don't have anything close to that yet, and I can't see anything short of that being a good indicator that we are close to AGI.

nopinsight 9y ago
For subgoals, there have been a lot of research results on that, mostly in the symbolic AI tradition. Even if an AI does not determine its own ultimate goal(s), it could still cause us a lot harm by creating subgoals that do not align with human values, which no one can articulate in full yet. Overall, the controversy is worth it because the force for progress is strong enough that another AI winter is highly unlikely if useful applications continue to be developed. Without the warnings, research on Provably Safe AI might not have sufficient resources especially talented people working on it.
- unityByFreedom 9y ago
  > the force for progress is strong enough that another AI winter is highly unlikely if useful applications continue to be developed. Really disagree here. All it takes is a few well positioned people, like those at Nnaisense, to create companies promising AGI and sucking up investment dollars. We all know greed is rampant in finance, and AI tech is a big word in startups/innovation these days that can land big VC dollars. There is corruption/snake oil in tech too, and you'll find the greediest float around the most hyped tech. I think companies like Nnaisense are the wrong place to invest, and that there is much more practical work to be done to advance humanity.
  - pdimitar 9y ago
    I see a small controversy in your statements, please help me understand: > Sometimes I get frustrated by the "AGI is an existential risk" quotes like Musk's because I feel that's a distraction from some really beneficial applications of machine learning that will help save lives. While I absolutely and adamantly disagree and I can't ever agree that warnings must be dismissed as "distractions", I extract from your statement here that you want us to move forward to a true AI no matter what. But then... > ...and that there is much more practical work to be done to advance humanity. Right now AI looks like snake oil selling, you realize that, right? I stand fully behind the statement that we have tons of practical and increasingly urgent issues to solve here in the material Earth. Unless of course you're implying that the strife for inventing a true AI (=AGI) is a practical work. IMO it isn't. It's better than the medieval alchemy, but not by much.
    - unityByFreedom 9y ago
      > While I absolutely and adamantly disagree and I can't ever agree that warnings must be dismissed as "distractions", Warnings can be faked. Fear mongering can be lessened when people are informed on a subject. > I extract from your statement here that you want us to move forward to a true AI no matter what. Where in the devil did you get that? I said no such thing. I think investments towards building AGI are largely a waste of money. If people want to spend their money on that, that's fine. I won't. > Right now AI looks like snake oil selling, you realize that, right? AGI does, yes. And, for those who do not understand the difference between data science and AGI, then probably all "AI tech" seems like snake oil. But it isn't. It's already helping doctors detect cancer, among other major issues. > Unless of course you're implying that the strife for inventing a true AI (=AGI) is a practical work I'm not. I have no idea where you got that from my comments.

nopinsight 9y ago

For subgoals, there have been a lot of research results on that, mostly in the symbolic AI tradition. Even if an AI does not determine its own ultimate goal(s), it could still cause us a lot harm by creating subgoals that do not align with human values, which no one can articulate in full yet. Overall, the controversy is worth it because the force for progress is strong enough that another AI winter is highly unlikely if useful applications continue to be developed. Without the warnings, research on Provably Safe AI might not have sufficient resources especially talented people working on it.

unityByFreedom 9y ago
> the force for progress is strong enough that another AI winter is highly unlikely if useful applications continue to be developed. Really disagree here. All it takes is a few well positioned people, like those at Nnaisense, to create companies promising AGI and sucking up investment dollars. We all know greed is rampant in finance, and AI tech is a big word in startups/innovation these days that can land big VC dollars. There is corruption/snake oil in tech too, and you'll find the greediest float around the most hyped tech. I think companies like Nnaisense are the wrong place to invest, and that there is much more practical work to be done to advance humanity.
- pdimitar 9y ago
  I see a small controversy in your statements, please help me understand: > Sometimes I get frustrated by the "AGI is an existential risk" quotes like Musk's because I feel that's a distraction from some really beneficial applications of machine learning that will help save lives. While I absolutely and adamantly disagree and I can't ever agree that warnings must be dismissed as "distractions", I extract from your statement here that you want us to move forward to a true AI no matter what. But then... > ...and that there is much more practical work to be done to advance humanity. Right now AI looks like snake oil selling, you realize that, right? I stand fully behind the statement that we have tons of practical and increasingly urgent issues to solve here in the material Earth. Unless of course you're implying that the strife for inventing a true AI (=AGI) is a practical work. IMO it isn't. It's better than the medieval alchemy, but not by much.
  - unityByFreedom 9y ago
    > While I absolutely and adamantly disagree and I can't ever agree that warnings must be dismissed as "distractions", Warnings can be faked. Fear mongering can be lessened when people are informed on a subject. > I extract from your statement here that you want us to move forward to a true AI no matter what. Where in the devil did you get that? I said no such thing. I think investments towards building AGI are largely a waste of money. If people want to spend their money on that, that's fine. I won't. > Right now AI looks like snake oil selling, you realize that, right? AGI does, yes. And, for those who do not understand the difference between data science and AGI, then probably all "AI tech" seems like snake oil. But it isn't. It's already helping doctors detect cancer, among other major issues. > Unless of course you're implying that the strife for inventing a true AI (=AGI) is a practical work I'm not. I have no idea where you got that from my comments.