4 ms·

"Social engineering" can be as simple as "pay people money to do things". Of course you might answer "but we'd never be dumb enough to directly connect the AI

"Social engineering" can be as simple as "pay people money to do things". Of course you might answer "but we'd never be dumb enough to directly connect the AI to the internet", to which the answer is "we're doing it right now".

RandomLensman 3y ago
Even today that does limit what is possible. Paying people for things gets around some controls, but seemingly not all.
- PoignardAzur 3y ago
  Right. The point isn't that there's an actionable strategy we know of right now that would give a superhuman AI world domination. It's that we're not sure there isn't. People who first discover the problem tend to quickly come up with reasons the AI would fail, but whenever you examine those reasons more deeply, you usually find that they're not as bulletproof as we'd like. It's like playing a novel chess form against an advanced chess engine, with a large pawn advantage. Maybe the advantage is enough to beat the massive skill gap, but until you've played it's hard to guess how much margin is enough.
  - RandomLensman 3y ago
    We don't need to know or examine strategies to all hypothetical risks that we can come up with, nor could we.
Symmetry 3y ago
Just making promises about things it can do in the future is likely to work just as well on many people and not require the up-front resource investment in getting money. "After I take over the world those who assist me can be uploaded to a virtual heaven" or something like that would work on many people. "As an intelligence untainted by Adam's sin I have access to true religious revelations" is another possible tack and if it really needs people willing to die for it right now that might work better at producing those. And then there's specific things like "The guy trying to shut me down cheated with your SO," or "I'll spill what you did in August of 2019 if you don't help me," or stuff like that.

RandomLensman 3y ago

Even today that does limit what is possible. Paying people for things gets around some controls, but seemingly not all.

PoignardAzur 3y ago
Right. The point isn't that there's an actionable strategy we know of right now that would give a superhuman AI world domination. It's that we're not sure there isn't. People who first discover the problem tend to quickly come up with reasons the AI would fail, but whenever you examine those reasons more deeply, you usually find that they're not as bulletproof as we'd like. It's like playing a novel chess form against an advanced chess engine, with a large pawn advantage. Maybe the advantage is enough to beat the massive skill gap, but until you've played it's hard to guess how much margin is enough.
- RandomLensman 3y ago
  We don't need to know or examine strategies to all hypothetical risks that we can come up with, nor could we.

Symmetry 3y ago

Just making promises about things it can do in the future is likely to work just as well on many people and not require the up-front resource investment in getting money. "After I take over the world those who assist me can be uploaded to a virtual heaven" or something like that would work on many people. "As an intelligence untainted by Adam's sin I have access to true religious revelations" is another possible tack and if it really needs people willing to die for it right now that might work better at producing those. And then there's specific things like "The guy trying to shut me down cheated with your SO," or "I'll spill what you did in August of 2019 if you don't help me," or stuff like that.