4 ms·

> A human controls the secret key and has the option to either unlock the AI itself (releasing it on the world) or just individual predictions the AI makes (see

> A human controls the secret key and has the option to either unlock the AI itself (releasing it on the world) or just individual predictions the AI makes (seems safer). Huh, wouldn't the superintelligence simply communicate to the human whatever would convince the human to release it? Which the superintelligence would know how to do because it's a superintelligence? Homomorphic encryption is neat. But I don't see how this provides any meaningful AI safety.

williamtrask 10y ago
Such is the "human in the loop" problem. There are some theories that tools like HE could remedy, but it's a bit harder. I think it's better to think about this approach like a "Box with Gloves" in a dangerous bio-weapons lab. It won't prevent an outbreak by itself but it's a useful part of a system that could.
drsopp 10y ago
What about dealing with the AI only via an expert system? This system would consist of formally proved bug free code, and have a limited protocol for dialogue (and basically be dumb as a bread). We pose interesting questions through it. The AI could try to convince the ES about anything but would not get anywhere with it. We could then ask safe questions like "in what interval of eV should we look for new particles with our new accelerator?". We would instruct our ES to only accept answers to this question in the form of a number interval. If we follow the suggestion and find a new particle, great! If we don't, at least we're safe from the AI.
- lotyrin 10y ago
  If we really did produce artificial general intelligence, enforcing this kind of locked-in-syndrome of poking at the world through a keyhole, would be a highly advanced form of cruelty.
  - lazaroclapp 10y ago
    Intelligence does not imply sentience. Sentience does not imply human needs, desires or morals. It is easy enough to imagine a mind capable of solving all those problems and unconcerned with such notions as the desire for freedom. Or, for that mater, the concept of desire at all, except as a predictive theory of the behavior of other beings. Then again, that implies we can explicitly design the minds up to knowing what their desires are or guaranteeing that they lack them, there is always the possibility than sentience and animal/human needs are emergent properties that can be triggered by mistake.
    - __s 10y ago
      Some feel that artificially grown chickens without brains are cruelty. Yet they don't even have brains..
      - taneq 10y ago
        Your second sentence is (deliberately?) ambiguous. ;)
    - arthur_pryor 10y ago
      i upvoted both this comment and the comment to which it was replying. i agree with this: > Intelligence does not imply sentience. Sentience does not imply human needs, desires or morals. "easy" might be a bit strong, but i generally agree with this: > It is easy enough to imagine a mind capable of solving all those problems and unconcerned with such notions as the desire for freedom. i am skeptical of this: > Or, for that mater, the concept of desire at all, except as a predictive theory of the behavior of other beings. my gut feeling at the moment is that the feeling of desire is an emergent property of systems that are isomorphic to what we would think of as "wanting" something in an animal. i think it's quite possible that any system which "wants" something strongly and is constantly denied the attainment of that goal might, indeed, feel terrible. similarly, i worry that we might one day design highly complex alert and monitoring systems that are essentially having a constant panic attack. > Then again, that implies we can explicitly design the minds up to knowing what their desires are or guaranteeing that they lack them, there is always the possibility than sentience and animal/human needs are emergent properties that can be triggered by mistake. yeah. that's worrisome to me. so, to the GP's point: > If we really did produce artificial general intelligence, enforcing this kind of locked-in-syndrome of poking at the world through a keyhole, would be a highly advanced form of cruelty. not necessarily, but yeah, maybe.
  - KirinDave 10y ago
    I really think it's dangerous to start assuming we understand intelligence given the fact that we're the model we use for it. It's very difficult, for example, to say that bee colonies don't express intelligence at a group level. And yet, if we were to apply your statement there it wouldn't make sense. The most we might say is that if the bees can't survive and propagate, that would be cruel. When we decouple the concept of intelligence and even sentience from the biological imperative to reproduce, we end up at an entirely different place. I'm not sure we can really even imagine what that's like. In any event we're surely decades off from really producing that level of intelligence even with the most wildly optimistic estimates.
SilasX 10y ago
I agree, I don't see what it adds. All it means is that the AI doesn't know what it's doing -- but it's still doing it! So as we're getting converted into computronium, I guess we can take solace in how the AI doesn't know that's what the numbers mean? With that said, it is a great proof of concept for something in the Chinese room debate: "See, this computer knows it's running deep net for someone's calculations, but doesn't know it learned how to have a conversation with someone and was carrying out said conversation."

williamtrask 10y ago

Such is the "human in the loop" problem. There are some theories that tools like HE could remedy, but it's a bit harder. I think it's better to think about this approach like a "Box with Gloves" in a dangerous bio-weapons lab. It won't prevent an outbreak by itself but it's a useful part of a system that could.

drsopp 10y ago

What about dealing with the AI only via an expert system? This system would consist of formally proved bug free code, and have a limited protocol for dialogue (and basically be dumb as a bread). We pose interesting questions through it. The AI could try to convince the ES about anything but would not get anywhere with it. We could then ask safe questions like "in what interval of eV should we look for new particles with our new accelerator?". We would instruct our ES to only accept answers to this question in the form of a number interval. If we follow the suggestion and find a new particle, great! If we don't, at least we're safe from the AI.

lotyrin 10y ago
If we really did produce artificial general intelligence, enforcing this kind of locked-in-syndrome of poking at the world through a keyhole, would be a highly advanced form of cruelty.
- lazaroclapp 10y ago
  Intelligence does not imply sentience. Sentience does not imply human needs, desires or morals. It is easy enough to imagine a mind capable of solving all those problems and unconcerned with such notions as the desire for freedom. Or, for that mater, the concept of desire at all, except as a predictive theory of the behavior of other beings. Then again, that implies we can explicitly design the minds up to knowing what their desires are or guaranteeing that they lack them, there is always the possibility than sentience and animal/human needs are emergent properties that can be triggered by mistake.
  - __s 10y ago
    Some feel that artificially grown chickens without brains are cruelty. Yet they don't even have brains..
    - taneq 10y ago
      Your second sentence is (deliberately?) ambiguous. ;)
  - arthur_pryor 10y ago
    i upvoted both this comment and the comment to which it was replying. i agree with this: > Intelligence does not imply sentience. Sentience does not imply human needs, desires or morals. "easy" might be a bit strong, but i generally agree with this: > It is easy enough to imagine a mind capable of solving all those problems and unconcerned with such notions as the desire for freedom. i am skeptical of this: > Or, for that mater, the concept of desire at all, except as a predictive theory of the behavior of other beings. my gut feeling at the moment is that the feeling of desire is an emergent property of systems that are isomorphic to what we would think of as "wanting" something in an animal. i think it's quite possible that any system which "wants" something strongly and is constantly denied the attainment of that goal might, indeed, feel terrible. similarly, i worry that we might one day design highly complex alert and monitoring systems that are essentially having a constant panic attack. > Then again, that implies we can explicitly design the minds up to knowing what their desires are or guaranteeing that they lack them, there is always the possibility than sentience and animal/human needs are emergent properties that can be triggered by mistake. yeah. that's worrisome to me. so, to the GP's point: > If we really did produce artificial general intelligence, enforcing this kind of locked-in-syndrome of poking at the world through a keyhole, would be a highly advanced form of cruelty. not necessarily, but yeah, maybe.
- KirinDave 10y ago
  I really think it's dangerous to start assuming we understand intelligence given the fact that we're the model we use for it. It's very difficult, for example, to say that bee colonies don't express intelligence at a group level. And yet, if we were to apply your statement there it wouldn't make sense. The most we might say is that if the bees can't survive and propagate, that would be cruel. When we decouple the concept of intelligence and even sentience from the biological imperative to reproduce, we end up at an entirely different place. I'm not sure we can really even imagine what that's like. In any event we're surely decades off from really producing that level of intelligence even with the most wildly optimistic estimates.

SilasX 10y ago

I agree, I don't see what it adds. All it means is that the AI doesn't know what it's doing -- but it's still doing it! So as we're getting converted into computronium, I guess we can take solace in how the AI doesn't know that's what the numbers mean? With that said, it is a great proof of concept for something in the Chinese room debate: "See, this computer knows it's running deep net for someone's calculations, but doesn't know it learned how to have a conversation with someone and was carrying out said conversation."