3 ms·

> If demon-summoning circles are going to be a thing that exists, I'd much rather have them in every household than have them used exclusively by the rich and p

> If demon-summoning circles are going to be a thing that exists, I'd much rather have them in every household than have them used exclusively by the rich and powerful. I generally agree with "democratization" of technology but this is one that should be available to as few people as possible (ideally 0). Treat it like a biological weapon or a nuclear stockpile, the area of effect will be so large that it doesn't really matter who uses it or who they point it at; everyone ends up suffering once it is activated. This is true under both limited tool-AI (under human command, but powerful and weaponized) and existential risk paperclip AI scenarios. A disappointing number of otherwise intelligent people are unwilling or incapable of seeing how this technology will be able to inflict mass human suffering and death.

selfhoster11 4y ago
The time to stop that would have been the early 1940s. Once you invent computers, that's it.
Schroedingersat 4y ago
Giving it exclusively to the most evil and cretinous money addicts is much worse than giving it to everyone.
- flangola7 4y ago
  Why is it worse for AI and not for nukes?
  - JohnFen 4y ago
    I understand that we probably disagree about this, but I don't think AI poses risks that are even remotely in the ballpark of the risks of nukes. If my neighbor has a nuke, he can (accidentally or deliberately) take out everyone around him with ease. AI can't.
    - flangola7 4y ago
      At this exact moment? No. But what about when it gets used for computer worms, trained on malicious hacking techniques, and it spreads through networks around the globe leaving a wake of destroyed infrastructure? This could happen as soon as this year. Modern defenses cannot stop dedicated offensive teams and now those abilities can be automated and scaled. This could be launched by your neighbor. Or how about slightly later when a multimodal model is trained to run an autonomous weapon system? Anyone with access to a metalshop and electric motors will be able to produce terminator-like killing machines. Robotics still has some distance to go but it's probably good enough to run a gymbal turret with superhuman accuracy and speed. A small previously insignificant nation or even organization could retrofit existing platforms with such technology - guns that don't miss, missiles that ignore flares and chaff, and point defenses that shoot incoming projectiles out of the sky, and use it to steamroll much larger powers. Not to mention the power of persuasive communication itself. Tyrants are not indestructible dragons, they just know to talk humans into killing other humans. I'm half sleep and even I can see that these possibilities are already here, much less what will be possible in coming months and years.
      - JohnFen 4y ago
        > Modern defenses cannot stop dedicated offensive teams and now those abilities can be automated and scaled. This could be launched by your neighbor. That's already been true for years (aside from my neighbors being able to do it at an effective scale), no AI is needed. And AI wouldn't make it easier for my neighbors to develop this sort of machinery. So, in my view, that problem is orthogonal to AI. > Anyone with access to a metalshop and electric motors will be able to produce terminator-like killing machines. I don't think the thing that makes it difficult for my neighbor to build an effective autonomous killing machine is AI. You could build a very effective one using basic heuristics. The limitation is about actually building the machinery. So I don't think AI is a game-changer here, either. Also, you're implying AGI here -- which is something that doesn't exist, and is very unlikely to exist anytime soon. > guns that don't miss, missiles that ignore flares and chaff, and point defenses that shoot incoming projectiles out of the sky, and use it to steamroll much larger powers. Also something that can be done right now, outside of being completely infallible. AI won't bring infallibility with it, so I'm not seeing how that changes anything. But here, you're no longer talking about individual action anyway, so it's a bit off-topic. You're talking about governmental or corporate action. > Not to mention the power of persuasive communication itself. Yep, current systems are clearly amplifiers of this. But even if you're 100% right in your predictions, none of those things are nearly as bad as everyone having their own nuke.

selfhoster11 4y ago

The time to stop that would have been the early 1940s. Once you invent computers, that's it.

Schroedingersat 4y ago

Giving it exclusively to the most evil and cretinous money addicts is much worse than giving it to everyone.

flangola7 4y ago
Why is it worse for AI and not for nukes?
- JohnFen 4y ago
  I understand that we probably disagree about this, but I don't think AI poses risks that are even remotely in the ballpark of the risks of nukes. If my neighbor has a nuke, he can (accidentally or deliberately) take out everyone around him with ease. AI can't.
  - flangola7 4y ago
    At this exact moment? No. But what about when it gets used for computer worms, trained on malicious hacking techniques, and it spreads through networks around the globe leaving a wake of destroyed infrastructure? This could happen as soon as this year. Modern defenses cannot stop dedicated offensive teams and now those abilities can be automated and scaled. This could be launched by your neighbor. Or how about slightly later when a multimodal model is trained to run an autonomous weapon system? Anyone with access to a metalshop and electric motors will be able to produce terminator-like killing machines. Robotics still has some distance to go but it's probably good enough to run a gymbal turret with superhuman accuracy and speed. A small previously insignificant nation or even organization could retrofit existing platforms with such technology - guns that don't miss, missiles that ignore flares and chaff, and point defenses that shoot incoming projectiles out of the sky, and use it to steamroll much larger powers. Not to mention the power of persuasive communication itself. Tyrants are not indestructible dragons, they just know to talk humans into killing other humans. I'm half sleep and even I can see that these possibilities are already here, much less what will be possible in coming months and years.
    - JohnFen 4y ago
      > Modern defenses cannot stop dedicated offensive teams and now those abilities can be automated and scaled. This could be launched by your neighbor. That's already been true for years (aside from my neighbors being able to do it at an effective scale), no AI is needed. And AI wouldn't make it easier for my neighbors to develop this sort of machinery. So, in my view, that problem is orthogonal to AI. > Anyone with access to a metalshop and electric motors will be able to produce terminator-like killing machines. I don't think the thing that makes it difficult for my neighbor to build an effective autonomous killing machine is AI. You could build a very effective one using basic heuristics. The limitation is about actually building the machinery. So I don't think AI is a game-changer here, either. Also, you're implying AGI here -- which is something that doesn't exist, and is very unlikely to exist anytime soon. > guns that don't miss, missiles that ignore flares and chaff, and point defenses that shoot incoming projectiles out of the sky, and use it to steamroll much larger powers. Also something that can be done right now, outside of being completely infallible. AI won't bring infallibility with it, so I'm not seeing how that changes anything. But here, you're no longer talking about individual action anyway, so it's a bit off-topic. You're talking about governmental or corporate action. > Not to mention the power of persuasive communication itself. Yep, current systems are clearly amplifiers of this. But even if you're 100% right in your predictions, none of those things are nearly as bad as everyone having their own nuke.