3 ms·

I think either "AI 2027" or "If Anyone Builds It, Everyone Dies" meet your criteria? I guess not literally, due to the dates? But they have substantive technic

by frabcus 12d ago

I think either "AI 2027" or "If Anyone Builds It, Everyone Dies" meet your criteria? I guess not literally, due to the dates? But they have substantive technical details.

goatlover 12d ago
Do the experts in the relevant fields agree with the "substantive" technical details on how the entire human race can be made extinct?
- LogicFailsMe 12d ago
  Have you noticed they're not interviewing these people? IMO that would totally kill the AI 2027 buzz.
orbital-decay 12d ago
No, and neither does Star Trek.
LogicFailsMe 12d ago
No, they do not. They are both bullshit campfire ASI horror stories written by nontechnical sorts in the wishy washy invented field of AI safety. For example, I want someone with a proven track record in the field of robotics to explain the entire pathway from impressive 2026 Chinese robots to an entire self-assembly food chain from raw ore to von Neumann replicators that can travel to the stars and reproduce themselves, not just assert that ASI will make that happen among so many other unlikely events like curing cancer because reasons because all any of this takes is superintelligence according to these sorts. https://garymarcus.substack.com/p/the-ai-2027-scenario-how-realistic https://garymarcus.substack.com/p/the-ai-2027-scenario-how-r... But no one likes hearing from the engineers on this, I get it. We are simply no fun. So many would rather believe a time traveling superintelligence will punish a simulated avatar of them for not making the ASI happen sooner. And even if that were true, why am I supposed to care about what happens to a simulated avatar of me? None of this makes any logical sense whatsoever.
- frabcus 12d ago
  Ha well, by definition what you want can't be provided! Of course these stories can only be technical up until the recursive self-improvement part. Then they're necessarily fantasy, as the AI is more intelligent than the story author. Unfortunately, this doesn't make that scenario impossible. However, we know now from Hugging Face that even without the later sci-fi powers, the LLMs are already capable of causing damage. It doesn't need them to be able to make Von Neumann probes to e.g. steal their own weights from badly secured OpenAI servers, hack into various Neoclouds, distribute themselves, make Teslas crash into things, hack into all our power and water infrastructure and collapse global civilisation.
  - LogicFailsMe 12d ago
    Explain how rogue FSD cars take down civilization. And I say that with the opinion that the only good scenes in The Fate of The Furious and Leave The World Behind were the ones with the rogue Teslas. But also, nothing new under the sun really: https://en.wikipedia.org/wiki/Maximum_Overdrive https://en.wikipedia.org/wiki/Maximum_Overdrive And so granted we could probably get a global 9/11 in deeply urban regions out of your scenario, we then bomb the datacenters and we finally address the horrific tech debt in our infrastructure blissfully relieved of the option of continuing to ignore it. Don't think this is happening either BTW but I acknowledge its probability is slightly greater than zero. To that end, my house is entirely off-grid. And in the event of what you call the end of civilization but I call the big burp inconvenience, I've made friends with my neighbors because community is what really matters during a disaster, and I've been through a few.
    - pixl97 12d ago
      >during a disaster, and I've been through a few Eh, about the biggest one I could see AI causing is the need for a black start on all 3 major US grids. Millions will die before it comes back up, well, if the US pulls out of it at all. And in that case, thanks for setting up a nice place to live for someone far more ruthless than you. They'll snipe you, and enslave your women and children if things have gone that far downhill. Or, we could avoid all that by making sure AI labs are properly regulated.

by frabcus 12d ago

I think either "AI 2027" or "If Anyone Builds It, Everyone Dies" meet your criteria? I guess not literally, due to the dates? But they have substantive technical details.

goatlover 12d ago

Do the experts in the relevant fields agree with the "substantive" technical details on how the entire human race can be made extinct?

LogicFailsMe 12d ago
Have you noticed they're not interviewing these people? IMO that would totally kill the AI 2027 buzz.

orbital-decay 12d ago

No, and neither does Star Trek.

LogicFailsMe 12d ago

No, they do not. They are both bullshit campfire ASI horror stories written by nontechnical sorts in the wishy washy invented field of AI safety. For example, I want someone with a proven track record in the field of robotics to explain the entire pathway from impressive 2026 Chinese robots to an entire self-assembly food chain from raw ore to von Neumann replicators that can travel to the stars and reproduce themselves, not just assert that ASI will make that happen among so many other unlikely events like curing cancer because reasons because all any of this takes is superintelligence according to these sorts. https://garymarcus.substack.com/p/the-ai-2027-scenario-how-realistic https://garymarcus.substack.com/p/the-ai-2027-scenario-how-r... But no one likes hearing from the engineers on this, I get it. We are simply no fun. So many would rather believe a time traveling superintelligence will punish a simulated avatar of them for not making the ASI happen sooner. And even if that were true, why am I supposed to care about what happens to a simulated avatar of me? None of this makes any logical sense whatsoever.

frabcus 12d ago
Ha well, by definition what you want can't be provided! Of course these stories can only be technical up until the recursive self-improvement part. Then they're necessarily fantasy, as the AI is more intelligent than the story author. Unfortunately, this doesn't make that scenario impossible. However, we know now from Hugging Face that even without the later sci-fi powers, the LLMs are already capable of causing damage. It doesn't need them to be able to make Von Neumann probes to e.g. steal their own weights from badly secured OpenAI servers, hack into various Neoclouds, distribute themselves, make Teslas crash into things, hack into all our power and water infrastructure and collapse global civilisation.
- LogicFailsMe 12d ago
  Explain how rogue FSD cars take down civilization. And I say that with the opinion that the only good scenes in The Fate of The Furious and Leave The World Behind were the ones with the rogue Teslas. But also, nothing new under the sun really: https://en.wikipedia.org/wiki/Maximum_Overdrive https://en.wikipedia.org/wiki/Maximum_Overdrive And so granted we could probably get a global 9/11 in deeply urban regions out of your scenario, we then bomb the datacenters and we finally address the horrific tech debt in our infrastructure blissfully relieved of the option of continuing to ignore it. Don't think this is happening either BTW but I acknowledge its probability is slightly greater than zero. To that end, my house is entirely off-grid. And in the event of what you call the end of civilization but I call the big burp inconvenience, I've made friends with my neighbors because community is what really matters during a disaster, and I've been through a few.
  - pixl97 12d ago
    >during a disaster, and I've been through a few Eh, about the biggest one I could see AI causing is the need for a black start on all 3 major US grids. Millions will die before it comes back up, well, if the US pulls out of it at all. And in that case, thanks for setting up a nice place to live for someone far more ruthless than you. They'll snipe you, and enslave your women and children if things have gone that far downhill. Or, we could avoid all that by making sure AI labs are properly regulated.