4 ms·

Please don't neglect to mention that a large number of very accomplished AI researchers share their concerns, or at the very least do not believe they should be

by emtel 3y ago

Please don't neglect to mention that a large number of very accomplished AI researchers share their concerns, or at the very least do not believe they should be dismissed out of hand.

deafpolygon 3y ago
I'm reading that a good number of those accomplished researchers that work at these companies, that stand to gain if they ensure that their company remains at the helm of AI research. Better that than introduce competition and unpredictable results, possibly leaving them behind in a highly competitive industry with no moat.
- kalkin 3y ago
  Are there messengers for the AI risk case who you'd consider credible, or is the message so outlandish in your mind that it doesn't really matter who's delivering it?
somesortofthing 3y ago
Researchers aren't immune to the incentives the rest of the field has. They're every bit as financially incentivized to push the AI risk narrative as the rest of the industry.
- acallaha 3y ago
  To name two Turing award winners: are you asserting that Geoff Hinton and Yoshua Bengio are pushing the AI risk narrative primarily because they're compromised by financial incentives? Geoff Hinton /left his job/ at Google because he sees the risks as real, so I think that's a tough case to make.
  - JumpCrisscross 3y ago
    > are you asserting that Geoff Hinton and Yoshua Bengio are pushing the AI risk narrative primarily because they're compromised by financial incentives Their message is being amplified and distorted by forces that are. That they're both comfortably wealthy from creating the problems they now preach solutions to is also no small matter.
  - worik 3y ago
    > Geoff Hinton /left his job/ at Google because he sees the risks as real, so I think that's a tough case to make. He did not leave his job when the bias and harm of the existing I models at Google were doing. He did not quit his job when Google fired their AI ethics champion rather than change their behaviour Has he had anything to say about the harm Google's algorithms do?
    - torginus 3y ago
      At least that's what he says. Maybe he got an unspecified offer to be the high priest of a to-be influential 'AI safety' organization. In my opinion, just like with the Patriot Act we are seeing a new world order trying to come into being. Come a decade, the few who bother and can learn about what transpires in the near future, will be shaking their heads at present humanity, asking how could we be so stupid and naive.
  - calf 3y ago
    I don't like the prevailing cynicism of dismissing the content of the message just because some bad/prejudiced/wrong factions are involved in it. It's not the way I learned to do research and answer questions. It's nevertheless an ad hominem to dismiss an issue just because some people out there are trying to co-opt it. There's a psychological explanation for that cynical behavior: internalized oppression. The focus is put on the morality of the people saying a thing, rather than just looking at the thing and deciding, independent of the existence of those groups, the objective, scientific implications. If it were a perfect society and people discovered deep neural networks, there would be no excuse to blame bad actors for posing the question of whether AGI is an existential problem or not, and what to do about it. Unlike bad groups of people, the question won't go away. In the real world, it is entirely possible to have to consider multiple issues at once, not at the expense of any issue.
- lumenwrites 3y ago
  Yudkowsky and many others have been working on this problem for decades, way before there was any financial incentive to speak of. The incentive here is that they want to live, and they want their loved ones to live.
  - mola 3y ago
    Yeah and they finally get their 15 minutes. Obviously they'll take it...
- emtel 3y ago
  Are there any previous cases you can think of where an industry exaggerated the dangers of their technology in order to increase profits? It seems to me that, if that is what is going on, its the first time this strategy has ever been attempted. I cannot think of a single case where manufacturers of cars, prescription drugs, firearms, chemicals, social media, airplanes, or nuclear power plants have exaggerated the dangers of their technology. In fact its common (perhaps universal) to downplay the dangers. Perhaps the simplest explanation for why researchers are saying there is a danger is that they genuinely believe it.
  - somesortofthing 3y ago
    They're exaggerating the efficacy of their technology, and are demanding a regulatory moat that currently doesn't exist under the pretext of the danger of that technology being misused. Notice the singular focus on "extinction risk" rather than flaws that the technology actually has right now, like hallucination, training bias, and prompt injection or the looming harm of AI adoption in both putting people out of work while substantially reducing the quality of anything automated with it. Part of it is marketing and part of it is ego on the part of people who grew up on fictional narratives about AI destroying the world and want to feel like they wield the power to make it do that, but whether any individual's belief is sincere or not, the overall push is a product of the massive incentives everyone involved in AI has to promote their product and entrench their market position.
    - emtel 3y ago
      You're saying that AI researchers are not immune to the power of incentives. I'm saying that there is no evidence that this sort of incentive has ever caused this sort of behavior before.
      - Planktonne 3y ago
        The Catholic church selling indulgences: hype the potential problems, profit from the panic.
rurp 3y ago
If they really believe that this is a significant risk then they are being incredibly reckless by not focusing all of their energy to halt AI development until a solution is found. Right now the opposite is happening, many researchers and companies who express these concerns are still barrelling ahead. A small group of people risking the lives of billions without their consent is morally repugnant.
- kalkin 3y ago
  Many of the signatories of this statement did also sign the statement calling for a halt to big training runs earlier this year.

by emtel 3y ago

Please don't neglect to mention that a large number of very accomplished AI researchers share their concerns, or at the very least do not believe they should be dismissed out of hand.

deafpolygon 3y ago

I'm reading that a good number of those accomplished researchers that work at these companies, that stand to gain if they ensure that their company remains at the helm of AI research. Better that than introduce competition and unpredictable results, possibly leaving them behind in a highly competitive industry with no moat.

kalkin 3y ago
Are there messengers for the AI risk case who you'd consider credible, or is the message so outlandish in your mind that it doesn't really matter who's delivering it?

somesortofthing 3y ago

Researchers aren't immune to the incentives the rest of the field has. They're every bit as financially incentivized to push the AI risk narrative as the rest of the industry.

acallaha 3y ago
To name two Turing award winners: are you asserting that Geoff Hinton and Yoshua Bengio are pushing the AI risk narrative primarily because they're compromised by financial incentives? Geoff Hinton /left his job/ at Google because he sees the risks as real, so I think that's a tough case to make.
- JumpCrisscross 3y ago
  > are you asserting that Geoff Hinton and Yoshua Bengio are pushing the AI risk narrative primarily because they're compromised by financial incentives Their message is being amplified and distorted by forces that are. That they're both comfortably wealthy from creating the problems they now preach solutions to is also no small matter.
- worik 3y ago
  > Geoff Hinton /left his job/ at Google because he sees the risks as real, so I think that's a tough case to make. He did not leave his job when the bias and harm of the existing I models at Google were doing. He did not quit his job when Google fired their AI ethics champion rather than change their behaviour Has he had anything to say about the harm Google's algorithms do?
  - torginus 3y ago
    At least that's what he says. Maybe he got an unspecified offer to be the high priest of a to-be influential 'AI safety' organization. In my opinion, just like with the Patriot Act we are seeing a new world order trying to come into being. Come a decade, the few who bother and can learn about what transpires in the near future, will be shaking their heads at present humanity, asking how could we be so stupid and naive.
- calf 3y ago
  I don't like the prevailing cynicism of dismissing the content of the message just because some bad/prejudiced/wrong factions are involved in it. It's not the way I learned to do research and answer questions. It's nevertheless an ad hominem to dismiss an issue just because some people out there are trying to co-opt it. There's a psychological explanation for that cynical behavior: internalized oppression. The focus is put on the morality of the people saying a thing, rather than just looking at the thing and deciding, independent of the existence of those groups, the objective, scientific implications. If it were a perfect society and people discovered deep neural networks, there would be no excuse to blame bad actors for posing the question of whether AGI is an existential problem or not, and what to do about it. Unlike bad groups of people, the question won't go away. In the real world, it is entirely possible to have to consider multiple issues at once, not at the expense of any issue.
lumenwrites 3y ago
Yudkowsky and many others have been working on this problem for decades, way before there was any financial incentive to speak of. The incentive here is that they want to live, and they want their loved ones to live.
- mola 3y ago
  Yeah and they finally get their 15 minutes. Obviously they'll take it...
emtel 3y ago
Are there any previous cases you can think of where an industry exaggerated the dangers of their technology in order to increase profits? It seems to me that, if that is what is going on, its the first time this strategy has ever been attempted. I cannot think of a single case where manufacturers of cars, prescription drugs, firearms, chemicals, social media, airplanes, or nuclear power plants have exaggerated the dangers of their technology. In fact its common (perhaps universal) to downplay the dangers. Perhaps the simplest explanation for why researchers are saying there is a danger is that they genuinely believe it.
- somesortofthing 3y ago
  They're exaggerating the efficacy of their technology, and are demanding a regulatory moat that currently doesn't exist under the pretext of the danger of that technology being misused. Notice the singular focus on "extinction risk" rather than flaws that the technology actually has right now, like hallucination, training bias, and prompt injection or the looming harm of AI adoption in both putting people out of work while substantially reducing the quality of anything automated with it. Part of it is marketing and part of it is ego on the part of people who grew up on fictional narratives about AI destroying the world and want to feel like they wield the power to make it do that, but whether any individual's belief is sincere or not, the overall push is a product of the massive incentives everyone involved in AI has to promote their product and entrench their market position.
  - emtel 3y ago
    You're saying that AI researchers are not immune to the power of incentives. I'm saying that there is no evidence that this sort of incentive has ever caused this sort of behavior before.
    - Planktonne 3y ago
      The Catholic church selling indulgences: hype the potential problems, profit from the panic.

rurp 3y ago

If they really believe that this is a significant risk then they are being incredibly reckless by not focusing all of their energy to halt AI development until a solution is found. Right now the opposite is happening, many researchers and companies who express these concerns are still barrelling ahead. A small group of people risking the lives of billions without their consent is morally repugnant.

kalkin 3y ago
Many of the signatories of this statement did also sign the statement calling for a halt to big training runs earlier this year.