4 ms·

If people make friendly AIs first, then we might be able to use those friendly AIs to figure out the actions that unfriendly AIs would take to get power, and cl

If people make friendly AIs first, then we might be able to use those friendly AIs to figure out the actions that unfriendly AIs would take to get power, and close them off first.

skybrian 9y ago
Okay, but do you need general AI for that? Why not narrow AI? On the defensive side I think there's a lot that can be done with narrow AI. Consider using really good penetration testers and bug finders to fix Internet security. For any given task, build a tool that just does that. It's like the least privilege principle in computer security.
- AgentME 9y ago
  I think there's a ton of routes to power for a general AI other than network security holes, and someone making a narrow AI just to fix all the bugs in our computer networks will miss nearly all of these. Consider an AI that makes a ton of money on mechanical turk -like jobs online, uses the money to run thousands of shell companies, over time hires the whole population to work on things related to paperclip factories, and crowds out other job opportunities so that anyone that goes against it goes jobless. Or an AI that figures out how to create cultural memes stupidly easily, gets people to vote in totalitarian governments, and then blackmails every single member of the central planning committees to optimize their countries for paperclip production. These ideas are mainly extreme proof-of-concept ideas, maybe there's ideas along these same lines that are much easier to pull off, but maybe these ideas as-is would be do-able for a possible superhuman intellect. I'm kind of thinking about what it looks like when individuals meticulously plan and then make tool-assisted speedrun videos through videogames, and then scaling it up to what it would look like if thousands of people worked together optimally for thousands of years brainstorming and calculating on how to speedrun through conquering a society. Physical limits permit this kind of computation as far as we know. Maybe this imagery is off-base, but I'm purposefully shooting high and worrying I'm not shooting high enough. Chimps would never even consider the possibility of "super-chimp" intelligences creating nuclear weapons. You can't make narrow AIs to defend from these possibilities if half the danger is our inability to even enumerate them.

If people make friendly AIs first, then we might be able to use those friendly AIs to figure out the actions that unfriendly AIs would take to get power, and close them off first.

skybrian 9y ago

Okay, but do you need general AI for that? Why not narrow AI? On the defensive side I think there's a lot that can be done with narrow AI. Consider using really good penetration testers and bug finders to fix Internet security. For any given task, build a tool that just does that. It's like the least privilege principle in computer security.

AgentME 9y ago
I think there's a ton of routes to power for a general AI other than network security holes, and someone making a narrow AI just to fix all the bugs in our computer networks will miss nearly all of these. Consider an AI that makes a ton of money on mechanical turk -like jobs online, uses the money to run thousands of shell companies, over time hires the whole population to work on things related to paperclip factories, and crowds out other job opportunities so that anyone that goes against it goes jobless. Or an AI that figures out how to create cultural memes stupidly easily, gets people to vote in totalitarian governments, and then blackmails every single member of the central planning committees to optimize their countries for paperclip production. These ideas are mainly extreme proof-of-concept ideas, maybe there's ideas along these same lines that are much easier to pull off, but maybe these ideas as-is would be do-able for a possible superhuman intellect. I'm kind of thinking about what it looks like when individuals meticulously plan and then make tool-assisted speedrun videos through videogames, and then scaling it up to what it would look like if thousands of people worked together optimally for thousands of years brainstorming and calculating on how to speedrun through conquering a society. Physical limits permit this kind of computation as far as we know. Maybe this imagery is off-base, but I'm purposefully shooting high and worrying I'm not shooting high enough. Chimps would never even consider the possibility of "super-chimp" intelligences creating nuclear weapons. You can't make narrow AIs to defend from these possibilities if half the danger is our inability to even enumerate them.