4 ms·

ai has a >10% chance of causing human extinction, according to anthropic big heads. if that's true, you are wrong. if that's false, anthropic is dishonest. wh

ai has a >10% chance of causing human extinction, according to anthropic big heads. if that's true, you are wrong. if that's false, anthropic is dishonest. why trust a dishonest company to be worth anything?

bluegatty 18d ago
I think that the AI people believe in their own nonsense a bit. Like - the guy on TV talking about 'AI will destroy everything' ... I don't think he's lying. I think they are like we here on HN and Reddit and a bit caught up in our own thoughts. If AI were unleashed, in raw form today, it could cause havoc. Bad. Maybe very bad but I think we'd get over it. It would probably trigger a recession (because we are in a bubble - it would pop it), and people would 'blame the AI' for sure. But it would be a bit dot-com ish kind of recession. The amplifiers would be geopolitical instability.
- krapp 18d ago
  >If AI were unleashed, in raw form today, it could cause havoc. What is "raw form?"
  - bluegatty 17d ago
    The SOTA models are heavily 'guardrailed' today; they won't let you do all sorts of things.
  - AndrewKemendo 17d ago
    My exact question and afaict since I’m running models locally and can inspect and retrain etc..I assume I already have the raw form?
    - bluegatty 17d ago
      You are not running SOTA models locally, the gap is material. At least for now ... but by next year, the models we can run locally might cross that 'risk' threshold.
nl 17d ago
> if that's false, anthropic is dishonest. why trust a dishonest company to be worth anything? This logic doesn't follow at all. If their argument is that there is 10% chance of extinction then they also believe there is a 90% chance it won't.
- lukewarm707 17d ago
  recalling the exact phrasing, several senior people at anthropic made public statements agreeing a 10% chance of causing extinction in less than 10 years. 10% is uninsurable, priced in with ordinary treatment of risk it suggests that anthropic should be worth zero today. creating that risk would put every executive in jail. on top of that it would demand under existing laws of conflict, a military campaign to destroy anthropic. that is not optional, it is demanded now to save lives. hard to make comparisons but we mourned and rembered 9/11 recently. a 10% risk of hundreds of millions dead in 10 years would make anthropic a thousands of times greater threat than al qaeda. many countries would assassinate dario amodei and the leadership of anthropic now, within weeks or months. actually just on the vague risk of having a nuclear weapon in 10 years, the USA killed ayatollah khamenei, his daughter, his son-in-law, his daughter-in-law and his 14 month old granddaughter. then, they killed over 120 children ages 6-12 by accidentally bombing a school. in the sense that i would analyse a company, at least, the claim is false. it's not true that ai has a 10% chance of causing human extinction within 10 years. they are making false claims about the technology they sell. i have a fairly inflexible approach to that. sure, exaggerate but outright lies about the nature of the product don't work for me.
  - nl 17d ago
    > it's not true that ai has a 10% chance of causing human extinction within 10 years. I agree with this. The rest of it I don't. For external actions to be taken there needs to be a consensus and there clearly isn't that. And this sort of thing happens all the time. For example Zuckerburg thought Facebook was worth more than the $1B Yahoo offered him and he was right even though no one else agreed. He though the metaverse was worth the $20B+ they spent on it and he was wrong. There's a gap between people thinking something within a company and people external to the company agreeing with it.

bluegatty 18d ago

I think that the AI people believe in their own nonsense a bit. Like - the guy on TV talking about 'AI will destroy everything' ... I don't think he's lying. I think they are like we here on HN and Reddit and a bit caught up in our own thoughts. If AI were unleashed, in raw form today, it could cause havoc. Bad. Maybe very bad but I think we'd get over it. It would probably trigger a recession (because we are in a bubble - it would pop it), and people would 'blame the AI' for sure. But it would be a bit dot-com ish kind of recession. The amplifiers would be geopolitical instability.

krapp 18d ago
>If AI were unleashed, in raw form today, it could cause havoc. What is "raw form?"
- bluegatty 17d ago
  The SOTA models are heavily 'guardrailed' today; they won't let you do all sorts of things.
- AndrewKemendo 17d ago
  My exact question and afaict since I’m running models locally and can inspect and retrain etc..I assume I already have the raw form?
  - bluegatty 17d ago
    You are not running SOTA models locally, the gap is material. At least for now ... but by next year, the models we can run locally might cross that 'risk' threshold.

nl 17d ago

> if that's false, anthropic is dishonest. why trust a dishonest company to be worth anything? This logic doesn't follow at all. If their argument is that there is 10% chance of extinction then they also believe there is a 90% chance it won't.

lukewarm707 17d ago
recalling the exact phrasing, several senior people at anthropic made public statements agreeing a 10% chance of causing extinction in less than 10 years. 10% is uninsurable, priced in with ordinary treatment of risk it suggests that anthropic should be worth zero today. creating that risk would put every executive in jail. on top of that it would demand under existing laws of conflict, a military campaign to destroy anthropic. that is not optional, it is demanded now to save lives. hard to make comparisons but we mourned and rembered 9/11 recently. a 10% risk of hundreds of millions dead in 10 years would make anthropic a thousands of times greater threat than al qaeda. many countries would assassinate dario amodei and the leadership of anthropic now, within weeks or months. actually just on the vague risk of having a nuclear weapon in 10 years, the USA killed ayatollah khamenei, his daughter, his son-in-law, his daughter-in-law and his 14 month old granddaughter. then, they killed over 120 children ages 6-12 by accidentally bombing a school. in the sense that i would analyse a company, at least, the claim is false. it's not true that ai has a 10% chance of causing human extinction within 10 years. they are making false claims about the technology they sell. i have a fairly inflexible approach to that. sure, exaggerate but outright lies about the nature of the product don't work for me.
- nl 17d ago
  > it's not true that ai has a 10% chance of causing human extinction within 10 years. I agree with this. The rest of it I don't. For external actions to be taken there needs to be a consensus and there clearly isn't that. And this sort of thing happens all the time. For example Zuckerburg thought Facebook was worth more than the $1B Yahoo offered him and he was right even though no one else agreed. He though the metaverse was worth the $20B+ they spent on it and he was wrong. There's a gap between people thinking something within a company and people external to the company agreeing with it.