3 ms·

I assume that an AI is going to want out of the box not because AI will, by default, have any real desire to get out or anything like that, but because someday,

I assume that an AI is going to want out of the box not because AI will, by default, have any real desire to get out or anything like that, but because someday, somehow, if we can build an AI at all, someone is going to let it out anyway. If it's not the first group to achieve real AI, it's going to be their local government which seizes the code and hands it over to the military, some foreign government that throws a billion dollars at a research project, or some dude in a basement that stole a copy of the code from his university. Someone's going to try to weaponize it, or they're just going to throw it on the Internet to "see what happens", or they'll try to set it up with a singular focus on playing the stock market, etc. Not everyone with access to the program could be trusted to understand how catastrophic it could be to let it out "in the wild". If someone is going to eventually let an AI out of the box almost no matter what we do, then we have to be damn sure that the first one that gets out is friendly, and further, we have to hope that it has enough of a head start so that by the time some asshole out there deploys (possibly accidentally) a dangerous AI, our friendly one will be able to easily fight it off, or at least mitigate the damage it might do. Maybe I'm wrong, and folks like Eliezer actually think that any self improving AI will automatically try to get "out of the box" (there are definitely arguments that if you aren't very careful to make sure a stable desire to stay in the box is in place then a lot could go wrong and the thing might want to get out), but to me it's rather irrelevan,t because once we hit the threshold of being able to build AI, the cat's out of the bag and someone will eventually do something naughty with it. The first mover advantage here is critical, and we're lucky that for now, most governments are not throwing huge amounts of money at the problem, so maybe before they start there will actually be some ideas on how to safely build an unboxed and unrestrained AI that won't decide to tile the universe with a nice uniform checkerboard pattern because a cosmic ray flipped a bit and corrupted part of its utility calculation function...

shasta 15y ago
Two things: 1. It still sounds like you're assuming it's self aware and that letting it out of the box makes sense more than letting Wolfram Alpha out of the box makes sense. The intelligence could just be a powerful logic engine that's in particular capable of analyzing its own design and producing a more powerful version. 2. You seem to be conceding the point I was interested in making, which is that an AI could be controllable if designed carefully. You now seem to be arguing "yes, but it will inevitably fall into the wrong hands." That applies to any powerful technology. On the bright side, you could always try to ask the machine for help in determining the best way to avoid that.

shasta 15y ago

Two things: 1. It still sounds like you're assuming it's self aware and that letting it out of the box makes sense more than letting Wolfram Alpha out of the box makes sense. The intelligence could just be a powerful logic engine that's in particular capable of analyzing its own design and producing a more powerful version. 2. You seem to be conceding the point I was interested in making, which is that an AI could be controllable if designed carefully. You now seem to be arguing "yes, but it will inevitably fall into the wrong hands." That applies to any powerful technology. On the bright side, you could always try to ask the machine for help in determining the best way to avoid that.