26 ms·

I keep hearing this refrain from AI doomsayers, but no one can ever tell me how these systems go from talking to the APIs they were hooked up with to somehow do

I keep hearing this refrain from AI doomsayers, but no one can ever tell me how these systems go from talking to the APIs they were hooked up with to somehow doing kinetic things that can kill humans. Your cheeseburger AI that logically decides humanity's continued existence is detrimental to the perfect cheeseburger: Even if it manages to make the jump from iterating on beef convection models to suggesting murder, what can it actually do? If it starts adding "kill yourself" to the end of the updated recipes it spits out, how is that any more dangerous than a dumb kid on Twitter?

cousin_it 11y ago
If the AI is allowed to print text that humans will read, that's just the standard AI-box experiment. Convince the assistant cook that you're a trapped benevolent AI and ask to be connected to the internet through their phone. Take over some machines via known exploits, hide, self-replicate, self-improve, find more exploits, get more machines, convince more people, game over. If you were trapped in a box with access to "just some APIs", and you could think 1000x faster than your jailers and could also self-improve, you'd find a way to get out and achieve whatever you consider to be your goals.
- Rodeoclash 11y ago
  http://www.yudkowsky.net/singularity/aibox/ http://www.yudkowsky.net/singularity/aibox/
Strilanc 11y ago
Here's a few basic reasons: a) There are business incentives to give the AI more capabilities and more information about the world. For example, if you have a cheeseburger bun factory with a complicated production line then you could use the AI to come up with more efficient layouts and machines for making cheeseburger buns. After awhile you realize that humans implementing what the AI explained is by far the slowest part of the pipeline, and ask if there's some way to have the AI build the production line by having it directly do the online ordering or 3d printing or whatever. At first you pay people to sanity-check what the AI is ordering and doing, but they never notice any problems so eventually you cut costs by laying them off. b) Side channels and exploits are a thing. Imagine yourself in the AI's place [1]. Is there really nothing you can do? Or is the security based around the fact that you don't try to escape? "Hey Larry, why is the description for so many orders coming out as 'Warning: ketchup parameter set too high. See <a href="javascript:{...sudo...WebSocket...dumps.wikimedia.org.../resetpassword/?user=billgates...}">the manual</a>.'?". "God damnit Ted, do you not see the link that says 'the manual'? Maybe you should click that before bothering me!" c) Instrumental goals. Any optimization process with an unqualified goal like "find a way to use this API to make lots of burgers, then do that" will favor plans where said process ends up with huge amounts of power over the world; simply because more burgers get made in that situation than in other situations. In some senses, failing to actively search for escape exploits could be seen as a design flaw because the algorithm is failing to find clever solutions that are better according to the stated goal. 1: http://lesswrong.com/lw/qk/that_alien_message/ http://lesswrong.com/lw/qk/that_alien_message/

cousin_it 11y ago

If the AI is allowed to print text that humans will read, that's just the standard AI-box experiment. Convince the assistant cook that you're a trapped benevolent AI and ask to be connected to the internet through their phone. Take over some machines via known exploits, hide, self-replicate, self-improve, find more exploits, get more machines, convince more people, game over. If you were trapped in a box with access to "just some APIs", and you could think 1000x faster than your jailers and could also self-improve, you'd find a way to get out and achieve whatever you consider to be your goals.

Rodeoclash 11y ago
http://www.yudkowsky.net/singularity/aibox/ http://www.yudkowsky.net/singularity/aibox/

Strilanc 11y ago

Here's a few basic reasons: a) There are business incentives to give the AI more capabilities and more information about the world. For example, if you have a cheeseburger bun factory with a complicated production line then you could use the AI to come up with more efficient layouts and machines for making cheeseburger buns. After awhile you realize that humans implementing what the AI explained is by far the slowest part of the pipeline, and ask if there's some way to have the AI build the production line by having it directly do the online ordering or 3d printing or whatever. At first you pay people to sanity-check what the AI is ordering and doing, but they never notice any problems so eventually you cut costs by laying them off. b) Side channels and exploits are a thing. Imagine yourself in the AI's place [1]. Is there really nothing you can do? Or is the security based around the fact that you don't try to escape? "Hey Larry, why is the description for so many orders coming out as 'Warning: ketchup parameter set too high. See <a href="javascript:{...sudo...WebSocket...dumps.wikimedia.org.../resetpassword/?user=billgates...}">the manual</a>.'?". "God damnit Ted, do you not see the link that says 'the manual'? Maybe you should click that before bothering me!" c) Instrumental goals. Any optimization process with an unqualified goal like "find a way to use this API to make lots of burgers, then do that" will favor plans where said process ends up with huge amounts of power over the world; simply because more burgers get made in that situation than in other situations. In some senses, failing to actively search for escape exploits could be seen as a design flaw because the algorithm is failing to find clever solutions that are better according to the stated goal. 1: http://lesswrong.com/lw/qk/that_alien_message/ http://lesswrong.com/lw/qk/that_alien_message/