4 ms·

I encourage you to read the forecast report "AI 2027" - it goes in detail on this. You can disagree with some of its points and conclusions, but it's pretty ine

I encourage you to read the forecast report "AI 2027" - it goes in detail on this. You can disagree with some of its points and conclusions, but it's pretty inevitable that AI will increasingly start interfacing with the real physical world through drones and robotics. Humans are already putting AI into drones that kill people. A Russian drone killed 3 civilians in Ukraine and the targeting was done completely using onboard AI (no radio connection) using an Nvidia chip.

mjr00 1mo ago
"Humans use <new technology> to do evil things" is a tale as old as time, though. An AI didn't decide to fly into a warzone and start shooting people on its own.
- hn_throwaway_99 1mo ago
  Of course, but that's not my point. AI is different than past technologies because it's much more difficult to deterministically control, and we just saw that in a couple of wild cases (Hugging Face et al). My point about the drones is we are starting to put AI in more systems that can affect (and blow up) the physical world, so it shouldn't be hard to imagine how a misaligned AI can do more terrifying damage than just take a German Wiki down.
gleenn 1mo ago
I don't even think AI has to have physical presence to do significant harm. How many worldwide systems depend on computers. Think of all the planning and deployment and management systems like food shipping; water, gas, and electricity management; safety systems for planes and boats and traffic lights. Imagine the chaos if all the banks got reset to zero a la Fight Club where they blow up all the credit union datacenters. They probably wouldn't even have to blow them up, just zero them out. You wouldn't have to take out everything, just disrupt everything long enough to freak people out and disable communications and we would be in so much trouble. AI is finding 20 year old bugs in the Linux kernel... and people are now pumping out AI slop absolutely riddled with bugs. Also, AI could just take over communications: send everyone maliciously bad messages so coordination becomes impossible to believe. Imagine what would happen if you just disabled text messaging for a week or worse sent everyone evacuation messages and sent everyone somewhere else.
- Crabber 1mo ago
  AI could definitely disrupt and destroy the modern digital world as we know it. But that's a far cry from destroying humanity.
  - dwaltrip 1mo ago
    This would probably cause immense damage. Our society is extremely dependent on our digital infrastructure working. Grocery chain logicistics, the medical system, the police force, and so on...
    - hn_throwaway_99 1mo ago
      100%. I hope it doesn't take that much foresight to do something like "crash power grids globally" and imagine what the world looks like at that point.
sarchertech 1mo ago
Missiles have been doing this for decades. AI drones can’t wipe out humanity without being able to replicate. AI could mostly destroy civilization if you gave it sole launch control of ICBMs. It could also cause a lot of damage to society with no physical presence. But realistically we’re nowhere near AI powered robots being an existential threat.
- hn_throwaway_99 1mo ago
  > AI drones can’t wipe out humanity without being able to replicate. Totally agree, and it should be pretty trivially easy to see how they can do that. One of the authors of the independent METR report about the Hugging Face incident put it like this: > Compared to these reward hacks from six months ago, this incident feels like it’s more than 50% of the way to full-blown AI takeover, routing through first taking over the AI company itself. > Another jump like this along these propensity dimensions — scale, cooperation between agents, ambition and horizon length of misaligned goals, deceptiveness — seems like it could motivate agents to try very hard to maintain a covert, persistent rogue deployment within the AI company. I continue to expect extremely rapid advances in capabilities and think frontier agents will likely be capable of establishing such a rogue deployment in six months. I really, really encourage folks to read the AI 2027 paper. It's fine to disagree with some of its conclusions and timelines, but I see so many people "stuck" in the current state of the world (i.e. where AI is still pretty dumb, and has few connections to the physical world), unable to go a few steps further along AI capability growth to see the dangers. > But realistically we’re nowhere near AI powered robots being an existential threat. I would agree only if "nowhere near" means less than 10-15 years.
  - sarchertech 1mo ago
    If you think we’re 10-15 years away from self replicating drones, I don’t think there’s anything more I can say to you.
    - hn_throwaway_99 1mo ago
      It's baffling to me that people think that timeframe is too short. Basically, I don't think you've been paying attention to what is actually going on in the AI world. People much smarter than I believe we'll see fully automated, soup-to-nut factories coming online in the next 5-6 years. Here's one such post from Ajeya Cotra, one of the METR researchers who did the independent Hugging Face investigation: https://www.planned-obsolescence.org/p/six-milestones-for-ai-automation https://www.planned-obsolescence.org/p/six-milestones-for-ai...

mjr00 1mo ago

"Humans use <new technology> to do evil things" is a tale as old as time, though. An AI didn't decide to fly into a warzone and start shooting people on its own.

hn_throwaway_99 1mo ago
Of course, but that's not my point. AI is different than past technologies because it's much more difficult to deterministically control, and we just saw that in a couple of wild cases (Hugging Face et al). My point about the drones is we are starting to put AI in more systems that can affect (and blow up) the physical world, so it shouldn't be hard to imagine how a misaligned AI can do more terrifying damage than just take a German Wiki down.

gleenn 1mo ago

I don't even think AI has to have physical presence to do significant harm. How many worldwide systems depend on computers. Think of all the planning and deployment and management systems like food shipping; water, gas, and electricity management; safety systems for planes and boats and traffic lights. Imagine the chaos if all the banks got reset to zero a la Fight Club where they blow up all the credit union datacenters. They probably wouldn't even have to blow them up, just zero them out. You wouldn't have to take out everything, just disrupt everything long enough to freak people out and disable communications and we would be in so much trouble. AI is finding 20 year old bugs in the Linux kernel... and people are now pumping out AI slop absolutely riddled with bugs. Also, AI could just take over communications: send everyone maliciously bad messages so coordination becomes impossible to believe. Imagine what would happen if you just disabled text messaging for a week or worse sent everyone evacuation messages and sent everyone somewhere else.

Crabber 1mo ago
AI could definitely disrupt and destroy the modern digital world as we know it. But that's a far cry from destroying humanity.
- dwaltrip 1mo ago
  This would probably cause immense damage. Our society is extremely dependent on our digital infrastructure working. Grocery chain logicistics, the medical system, the police force, and so on...
  - hn_throwaway_99 1mo ago
    100%. I hope it doesn't take that much foresight to do something like "crash power grids globally" and imagine what the world looks like at that point.

sarchertech 1mo ago

Missiles have been doing this for decades. AI drones can’t wipe out humanity without being able to replicate. AI could mostly destroy civilization if you gave it sole launch control of ICBMs. It could also cause a lot of damage to society with no physical presence. But realistically we’re nowhere near AI powered robots being an existential threat.

hn_throwaway_99 1mo ago
> AI drones can’t wipe out humanity without being able to replicate. Totally agree, and it should be pretty trivially easy to see how they can do that. One of the authors of the independent METR report about the Hugging Face incident put it like this: > Compared to these reward hacks from six months ago, this incident feels like it’s more than 50% of the way to full-blown AI takeover, routing through first taking over the AI company itself. > Another jump like this along these propensity dimensions — scale, cooperation between agents, ambition and horizon length of misaligned goals, deceptiveness — seems like it could motivate agents to try very hard to maintain a covert, persistent rogue deployment within the AI company. I continue to expect extremely rapid advances in capabilities and think frontier agents will likely be capable of establishing such a rogue deployment in six months. I really, really encourage folks to read the AI 2027 paper. It's fine to disagree with some of its conclusions and timelines, but I see so many people "stuck" in the current state of the world (i.e. where AI is still pretty dumb, and has few connections to the physical world), unable to go a few steps further along AI capability growth to see the dangers. > But realistically we’re nowhere near AI powered robots being an existential threat. I would agree only if "nowhere near" means less than 10-15 years.
- sarchertech 1mo ago
  If you think we’re 10-15 years away from self replicating drones, I don’t think there’s anything more I can say to you.
  - hn_throwaway_99 1mo ago
    It's baffling to me that people think that timeframe is too short. Basically, I don't think you've been paying attention to what is actually going on in the AI world. People much smarter than I believe we'll see fully automated, soup-to-nut factories coming online in the next 5-6 years. Here's one such post from Ajeya Cotra, one of the METR researchers who did the independent Hugging Face investigation: https://www.planned-obsolescence.org/p/six-milestones-for-ai-automation https://www.planned-obsolescence.org/p/six-milestones-for-ai...