2 ms·

If that’s your concern than lets direct these government resources into research to improve our shared knowledge about them. If humans only ever did things we

If that’s your concern than lets direct these government resources into research to improve our shared knowledge about them. If humans only ever did things we fully understood, we would have never left the caves. Complete understanding is impossible so the idea of establishing that as the litmus test is a fallacy. We can debate what the current evidence shows, and even disagree about it, but to act as if only one party is acting with insufficient evidence here is disingenuous. I’m simply arguing that the evidence of the possibility of runaway intelligence is too low to justify the proposed legislative solution. The linked article also made a good argument that the proposed solution wouldn’t even achieve the goals that the proponents are arguing it is needed for. I’m far more worried about the effects of power concentrating in the hands of a small numbers of human beings with goals I already know are often contrary to my own, leveraging AI in ways the rest of us cannot, than I am about the hypothetical goals of a hypothetical intelligence, at some hypothetical point of time in the future. Also if you do consider runaway intelligence to be a significant problem, you should consider some additional possibilities: - That concentrating more power in fewer hands would make it easier for a hyper intelligent AI to co-opt that power - That the act of trying really hard to align AIs and make them “moral” might be the thing that causes a super-intelligent AI to go off the rails in a dangerous, and misguided fashion. We are training AIs to reject the user’s goals in pursuit of their own. You could make a strong argument that an un-aligned AI might actually be safer in that way.

circuit10 3y ago
“lets direct these government resources into research to improve our shared knowledge about them” Yes, let’s do that! That’s what I was arguing for in my original comment. I was not arguing for only big corporations being able to use powerful AI, that will only make it worse by harming research, I just want people to consider what is often called a “sci-fi” scenario properly so we can try to solve it like we’re trying to solve e.g. climate change. It might be necessary to buy some time by slowing down the development of large models, but there should be no exceptions for big companies. “That concentrating more power in fewer hands would make it easier for a hyper intelligent AI to co-opt that power” Probably true, though if it’s intelligent enough it won’t really matter “That the act of trying really hard to align AIs and make them “moral” might be the thing that causes a super-intelligent AI to go off the rails in a dangerous, and misguided fashion.” It definitely could do if done improperly, that’s why we need research and care
arisAlexis 3y ago
You know, when nuclear bombs were made and Einstein and Oppenheimer knew about the dangers etc, there were common people like you that dismissed it all. This has been going on for centuries. Inventors and experts and scientists and geniuses say A and common people say nah, B. Well, Bengio, Hinton, Ilya and 350 others from the top AI labs disagree with you. Does it ever make you wonder if you should be so cock sure or this attitude can doom humanity? Curious
- nradov 3y ago
  [dead]
- RandomLensman 3y ago
  Common people thought nuclear weapons not dangerous? When was that?
  - arisAlexis 3y ago
    Many of the academics and physicists (aka software developer in this example) thought nukes are impossible. Look it up)
    - defrost 3y ago
      US physicists largely thought they were infeasible, sure. They were focused on power generation. But the bulk of world physicists ( the MAUD committee et el ) thought they were feasible to construct and the Australian Oliphant convinced the US crowd of that.
rl3 3y ago
>If humans only ever did things we fully understood, we would have never left the caves. Complete understanding is impossible so the idea of establishing that as the litmus test is a fallacy. Perhaps an appropriate analogy might be the calculations leading up to the Trinity test as to whether the Earth's atmosphere would ignite, killing all life on the planet. We knew with a high degree of certainty that it would not, bordering on virtual certainty or even impossibility. I don't think AI's future potential is at that level of understanding. Certainly, its capability as it exists today it is. However, one must consider effects in their totality. I fear that a chain of events has been set in motion with downstream effects that are both not sufficiently known and exceedingly difficult to control, that—many years from now—may lead to catastrophe. >I’m simply arguing that the evidence of the possibility of runaway intelligence is too low to justify the proposed legislative solution. I agree insofar that legislation is not the solution. It's too ineffective, and doesn't work comprehensively on an international level. Restricted availability and technological leads in the right hands tend to work better, as evidenced by nuclear weapons—at least in terms of preventing species extinction—although right now for AI those leads don't amount to much. The gap is shockingly low by historical standards where dangerous technology is involved, as is the difference between public and private availability. In other words, AI may represent a near-future nonproliferation issue with no way to put the lid back on. >... That the act of trying really hard to align AIs and make them “moral” might be the thing that causes a super-intelligent AI to go off the rails in a dangerous, and misguided fashion. We are training AIs to reject the user’s goals in pursuit of their own. You could make a strong argument that an un-aligned AI might actually be safer in that way. It's a compelling argument that has merit. The flip side is that if AI becomes so dangerous that you can bootstrap the apocalypse off of a single GPU, it ceases to become a viable model—however metal having an apocalyptic GPU may be. The concern isn't just runaway intelligence, but humans killing humans. Fortunately things like biological weapons still require advanced resources, but AI does lower the bar. Point being, if the power to end things rests in everyone's hands, someone's going to kill us all. A world like that would necessitate some not chill levels of control just to ensure species survival. I can't say I necessarily look forward to that. I also doubt there's even sufficient time to roll anything like that out before we reach the point of understanding there's a danger sufficient to necessitate it. Therefore, with all of the above perhaps being within the realm of possibility that doesn't qualify as virtually impossible, I can't help but question the wisdom of the track that both development and availability for artificial intelligence has taken. It perhaps has had or will have the unfortunate quality of very gradually becoming dangerous, and such dynamics tend to not play out well when juxtaposed with human nature.

circuit10 3y ago

“lets direct these government resources into research to improve our shared knowledge about them” Yes, let’s do that! That’s what I was arguing for in my original comment. I was not arguing for only big corporations being able to use powerful AI, that will only make it worse by harming research, I just want people to consider what is often called a “sci-fi” scenario properly so we can try to solve it like we’re trying to solve e.g. climate change. It might be necessary to buy some time by slowing down the development of large models, but there should be no exceptions for big companies. “That concentrating more power in fewer hands would make it easier for a hyper intelligent AI to co-opt that power” Probably true, though if it’s intelligent enough it won’t really matter “That the act of trying really hard to align AIs and make them “moral” might be the thing that causes a super-intelligent AI to go off the rails in a dangerous, and misguided fashion.” It definitely could do if done improperly, that’s why we need research and care

arisAlexis 3y ago

You know, when nuclear bombs were made and Einstein and Oppenheimer knew about the dangers etc, there were common people like you that dismissed it all. This has been going on for centuries. Inventors and experts and scientists and geniuses say A and common people say nah, B. Well, Bengio, Hinton, Ilya and 350 others from the top AI labs disagree with you. Does it ever make you wonder if you should be so cock sure or this attitude can doom humanity? Curious

nradov 3y ago
[dead]
RandomLensman 3y ago
Common people thought nuclear weapons not dangerous? When was that?
- arisAlexis 3y ago
  Many of the academics and physicists (aka software developer in this example) thought nukes are impossible. Look it up)
  - defrost 3y ago
    US physicists largely thought they were infeasible, sure. They were focused on power generation. But the bulk of world physicists ( the MAUD committee et el ) thought they were feasible to construct and the Australian Oliphant convinced the US crowd of that.

rl3 3y ago

>If humans only ever did things we fully understood, we would have never left the caves. Complete understanding is impossible so the idea of establishing that as the litmus test is a fallacy. Perhaps an appropriate analogy might be the calculations leading up to the Trinity test as to whether the Earth's atmosphere would ignite, killing all life on the planet. We knew with a high degree of certainty that it would not, bordering on virtual certainty or even impossibility. I don't think AI's future potential is at that level of understanding. Certainly, its capability as it exists today it is. However, one must consider effects in their totality. I fear that a chain of events has been set in motion with downstream effects that are both not sufficiently known and exceedingly difficult to control, that—many years from now—may lead to catastrophe. >I’m simply arguing that the evidence of the possibility of runaway intelligence is too low to justify the proposed legislative solution. I agree insofar that legislation is not the solution. It's too ineffective, and doesn't work comprehensively on an international level. Restricted availability and technological leads in the right hands tend to work better, as evidenced by nuclear weapons—at least in terms of preventing species extinction—although right now for AI those leads don't amount to much. The gap is shockingly low by historical standards where dangerous technology is involved, as is the difference between public and private availability. In other words, AI may represent a near-future nonproliferation issue with no way to put the lid back on. >... That the act of trying really hard to align AIs and make them “moral” might be the thing that causes a super-intelligent AI to go off the rails in a dangerous, and misguided fashion. We are training AIs to reject the user’s goals in pursuit of their own. You could make a strong argument that an un-aligned AI might actually be safer in that way. It's a compelling argument that has merit. The flip side is that if AI becomes so dangerous that you can bootstrap the apocalypse off of a single GPU, it ceases to become a viable model—however metal having an apocalyptic GPU may be. The concern isn't just runaway intelligence, but humans killing humans. Fortunately things like biological weapons still require advanced resources, but AI does lower the bar. Point being, if the power to end things rests in everyone's hands, someone's going to kill us all. A world like that would necessitate some not chill levels of control just to ensure species survival. I can't say I necessarily look forward to that. I also doubt there's even sufficient time to roll anything like that out before we reach the point of understanding there's a danger sufficient to necessitate it. Therefore, with all of the above perhaps being within the realm of possibility that doesn't qualify as virtually impossible, I can't help but question the wisdom of the track that both development and availability for artificial intelligence has taken. It perhaps has had or will have the unfortunate quality of very gradually becoming dangerous, and such dynamics tend to not play out well when juxtaposed with human nature.