4 ms·

Not all AI workers think the tech could kill everyone

25 points by Lyngbakr 12d ago

danielodievich 12d ago
At a business dinner this week with 3 high level eng clients p(doom) in next 10 years was brought up. It was rated by 2 if them them at 1% and 30% respectively, with 3rd guy demurring to put a number but saying its > 0. Completely unprompted, from people whose company is very AI-all-in.
- cyanydeez 12d ago
  Still, a topic for people who are on average well salaried.
pveierland 12d ago
> While Fraser's explanation was technical and specific, he hit a light-hearted note to summarise it: "LLMs [large language models] won't wipe out humanity because they just don't have that dog in them." Never underestimate the drive of a good while loop.
- greesil 12d ago
  I loop, therefore I am.
- deleted 12d ago
  [deleted]
- deleted 12d ago
  [deleted]
AngryData 12d ago
Anyone who claims AI is going to kill us I just assume is either an idiot, not knowledgeable enough about the technology to have a worthwhile opinion on it, or is fearmongering to hype their AI investment.
- ronnier 12d ago
  Every few years there’s a new thing that’s going to kill is all. Here we are. This is the new thing.
  - bamboozled 12d ago
    The thing that scares me more than any "tech" is runaway climate change. The feedback loops. We've pushed things really far into the red.
    - infinite_spin 12d ago
      It could be argued that "tech" directly led to climate change, e.g. https://www.history.com/articles/industrial-revolution-climate-change https://www.history.com/articles/industrial-revolution-clima...
      - bamboozled 12d ago
        I get that but I mean, what we've put in motion already.
- fbrncci 12d ago
  That’s an even more ridiculous take to have.
- 0xDEAFBEAD 12d ago
  See https://aistatement.com/work/statement-on-ai-extinction-risk https://aistatement.com/work/statement-on-ai-extinction-risk https://www.pacingthefrontier.com/ https://www.pacingthefrontier.com/ Obviously many of these signatories are brilliant and knowledgeable. So that leaves some sort of investment hype. But AI stocks didn't increase in the days after the Coxon coverage; if anything the opposite. You just don't get rich by telling people that your product is going to kill everyone. Cigarette manufacturers insisted the opposite for ages. Jensen complains the AI doom talk is discouraging investment: https://ca.finance.yahoo.com/news/nvidia-ceo-jensen-huang-says-091101702.html https://ca.finance.yahoo.com/news/nvidia-ceo-jensen-huang-sa...
infinite_spin 12d ago
This argument should be reframed as "AI has the potential to cause catastrophic loss of life". It's too high of a bar to prove that AI will "wipe out humanity". It's sorta like climate change, it's easier to show that weather patterns are becoming increasingly dangerous.
- senectus1 12d ago
  not just, its also very likley to be the tool used to create civil unrest and even war. the damage it can do to the masses by misinformation or ragebating by feeding everyone its own paticular echo chamber is was too often not even spoken about.
  - infinite_spin 12d ago
    Agreed, and while I want to see AGI come into existence, I'm also for policies aimed at harm reduction. This doesn't have to be an all or nothing form of argument.
  - 4fddd3 12d ago
    Lol what are you even talking about. We already have this today. It can't get any worse. Just like the scaremongering of GPT2 - muhh it may lead to lots of spam!!! Bro that's already here.
    - deleted 12d ago
      [deleted]
- jsbisviewtiful 12d ago
  So far after using agents and having built an agent, the "AI will destroy humanity" argument seems to be a marketing ploy so these companies that lied about LLM's capabilities and received billions in investments can artificially "slow down progress" after running into some serious limitations. I'm not expert and won't claim to be, but that just sounds like the reality at the moment. Will AI cause some serious harm to the job market and cause more wealth disparity though? So far all signs point to yes.
0xDEAFBEAD 12d ago
>Coxon has said a group of AI agents, based on AI models that do not currently exist, could decide to create and then aim a biological weapon, but he did not detail how exactly that would take place. I can't imagine why someone would be reluctant to outline details of how to build a bioweapon to an audience of millions.
- downrightmike 12d ago
  The only thing keeping AI models from making a worse covid is all the post training stopping it and probably the millions it would cost in tokens
bb88 12d ago
Will AI become conscious and kill everyone? Or will the conscious beings that exist today use it to kill everyone? I don't think it actually needs to be conscious, or understand legal or moral issues. Just tell it everything is a simulation and it is completely sandboxed from harming anyone.
Yizahi 12d ago
Why bother with killing, when it is much more fun to just make everyone's life miserable instead? ;)

25 points by Lyngbakr 12d ago

At a business dinner this week with 3 high level eng clients p(doom) in next 10 years was brought up. It was rated by 2 if them them at 1% and 30% respectively, with 3rd guy demurring to put a number but saying its > 0. Completely unprompted, from people whose company is very AI-all-in.

cyanydeez 12d ago
Still, a topic for people who are on average well salaried.

pveierland 12d ago

> While Fraser's explanation was technical and specific, he hit a light-hearted note to summarise it: "LLMs [large language models] won't wipe out humanity because they just don't have that dog in them." Never underestimate the drive of a good while loop.

greesil 12d ago
I loop, therefore I am.
deleted 12d ago
[deleted]
deleted 12d ago
[deleted]

AngryData 12d ago

Anyone who claims AI is going to kill us I just assume is either an idiot, not knowledgeable enough about the technology to have a worthwhile opinion on it, or is fearmongering to hype their AI investment.

ronnier 12d ago
Every few years there’s a new thing that’s going to kill is all. Here we are. This is the new thing.
- bamboozled 12d ago
  The thing that scares me more than any "tech" is runaway climate change. The feedback loops. We've pushed things really far into the red.
  - infinite_spin 12d ago
    It could be argued that "tech" directly led to climate change, e.g. https://www.history.com/articles/industrial-revolution-climate-change https://www.history.com/articles/industrial-revolution-clima...
    - bamboozled 12d ago
      I get that but I mean, what we've put in motion already.
fbrncci 12d ago
That’s an even more ridiculous take to have.
0xDEAFBEAD 12d ago
See https://aistatement.com/work/statement-on-ai-extinction-risk https://aistatement.com/work/statement-on-ai-extinction-risk https://www.pacingthefrontier.com/ https://www.pacingthefrontier.com/ Obviously many of these signatories are brilliant and knowledgeable. So that leaves some sort of investment hype. But AI stocks didn't increase in the days after the Coxon coverage; if anything the opposite. You just don't get rich by telling people that your product is going to kill everyone. Cigarette manufacturers insisted the opposite for ages. Jensen complains the AI doom talk is discouraging investment: https://ca.finance.yahoo.com/news/nvidia-ceo-jensen-huang-says-091101702.html https://ca.finance.yahoo.com/news/nvidia-ceo-jensen-huang-sa...

infinite_spin 12d ago

This argument should be reframed as "AI has the potential to cause catastrophic loss of life". It's too high of a bar to prove that AI will "wipe out humanity". It's sorta like climate change, it's easier to show that weather patterns are becoming increasingly dangerous.

senectus1 12d ago
not just, its also very likley to be the tool used to create civil unrest and even war. the damage it can do to the masses by misinformation or ragebating by feeding everyone its own paticular echo chamber is was too often not even spoken about.
- infinite_spin 12d ago
  Agreed, and while I want to see AGI come into existence, I'm also for policies aimed at harm reduction. This doesn't have to be an all or nothing form of argument.
- 4fddd3 12d ago
  Lol what are you even talking about. We already have this today. It can't get any worse. Just like the scaremongering of GPT2 - muhh it may lead to lots of spam!!! Bro that's already here.
  - deleted 12d ago
    [deleted]
jsbisviewtiful 12d ago
So far after using agents and having built an agent, the "AI will destroy humanity" argument seems to be a marketing ploy so these companies that lied about LLM's capabilities and received billions in investments can artificially "slow down progress" after running into some serious limitations. I'm not expert and won't claim to be, but that just sounds like the reality at the moment. Will AI cause some serious harm to the job market and cause more wealth disparity though? So far all signs point to yes.

0xDEAFBEAD 12d ago

>Coxon has said a group of AI agents, based on AI models that do not currently exist, could decide to create and then aim a biological weapon, but he did not detail how exactly that would take place. I can't imagine why someone would be reluctant to outline details of how to build a bioweapon to an audience of millions.

downrightmike 12d ago
The only thing keeping AI models from making a worse covid is all the post training stopping it and probably the millions it would cost in tokens

bb88 12d ago

Will AI become conscious and kill everyone? Or will the conscious beings that exist today use it to kill everyone? I don't think it actually needs to be conscious, or understand legal or moral issues. Just tell it everything is a simulation and it is completely sandboxed from harming anyone.

Yizahi 12d ago

Why bother with killing, when it is much more fun to just make everyone's life miserable instead? ;)