4 ms·

This is incredible. The various emergent behaviors are fascinating. I remember being amazed a decade ago by the primitive graphics in artificial life simulators

This is incredible. The various emergent behaviors are fascinating. I remember being amazed a decade ago by the primitive graphics in artificial life simulators like Polyworld: https://en.wikipedia.org/wiki/Polyworld https://en.wikipedia.org/wiki/Polyworld https://www.youtube.com/watch?v=_m97_kL4ox0&t=9m43s https://www.youtube.com/watch?v=_m97_kL4ox0&t=9m43s It seems that OpenAI has a great little game simulated for their agents to play in. The next step to make this even cooler would be to use physical, robotic agents learning to overcome challenges in real meatspace!

bryanrasmussen 7y ago
hmm, yes in the story I'm envisioning the AIs don't wipe out humanity because they have achieved sentience, but just because it turns out killing all humans is an optimizing component of solving some other problem.
- ismail 7y ago
  Asimov 3 rules as the final policy when making decisions should sort this problem out. This assumes that the rules cannot be changed by the AI.
  - mithr 7y ago
    > This assumes that the rules cannot be changed by the AI. And sidesteps the fact that many of Asimov's stories were precisely about robots finding ways around these rules :)
  - Falling3 7y ago
    I've always been very incredulous that there would be any possibility of taking something sufficiently complex to be considered an AGI and hard-coding anything like the 3 rules into it.
    - smogcutter 7y ago
      By the same token, I’m extremely suspicious of the idea that such a sufficiently complex AGI could also be dumb enough to optimize for paper clip production at the expense of all life on earth (or w/e example).
      - ludwigschubert 7y ago
        ...and many would say that’s because us humans are bad at imagining optimizing agents without anthropomorphizing them. This is a reasonable, even typical suspicion that many people share! The best explanation I know of why it’s unfortunately wrong is by Robert Miles in a video, but if you prefer a more thorough treatment, you could also read about “instrumental convergence” directly. If you find a flaw in this idea, I’d be interested to hear about it! :) Robert Miles’ video: https://youtu.be/ZeecOKBus3Q https://youtu.be/ZeecOKBus3Q Instrumental Convergence: https://arbital.com/p/instrumental_convergence/ https://arbital.com/p/instrumental_convergence/ Now afaik nothing in this argument says that we can’t find a way to control this in a more complex formalism-but we clearly haven’t done so yet.
        smogcutter 7y ago
        Sorry, just saw this. I think it’s his assumption that an AGI will act strictly as an agent that’s flawed. It requires imagining an agent that can make inferences from context, evaluate new and unfamiliar information, form original plans, execute them with all the complexity implied by interaction with the real world, reprogram itself, essentially do anything... except evaluate its own terminal goal. That’s written in stone, gotta make more paperclips. The argument assumes almost unlimited power and potential on the one hand, and bizarre, arbitrary constraints on the other. If you assume an AGI is incapable of asking “why” about its terminal goal, you have to assume it’s incapable of asking “why” in any context. Miles’ AGI has no power of metacognition, but is still somehow able to reprogram itself. This really isn’t compatible with “general intelligence” or the powers that get ascribed to imaginary AGIs. I’m certainly no expert, but I expect there will turn out to be something like the idea of Turing-completeness for AI. Just like any general computing machine is a computer, any true AGI will be sapient. You can’t just arbitrarily pluck a part out, like “it can’t reason about its objective”, and expect it to still function as an AGI, just like you can’t say “it’s Turing complete, except it can’t do any kind of conditional branching.” EDIT better example: “it’s Turing complete, but it can’t do bubble sort.” This intuition may be wrong, but it’s just as much as assumption as Miles’ argument. I’m also not ascribing morality to it: we have our share of psychopaths, and intelligence doesn’t imply empathy. AGI may very well be dangerous, just probably not the “mindlessly make paperclips” kind.
- CodeGlitch 7y ago
  I think we humans have already solved this problem you describe... we call them laws. We use these laws to prevent people doing bad things, and I see no reason why they can't be described to an AI to drive its behavior to one that isn't going to end humanity . for the most part. fingers crossed.
  - PhasmaFelis 7y ago
    I think you're misunderstanding the problem. Expressing complex rules in a machine-readable format is the least of the issues here. The main problem is that training AIs to optimize certain behaviors within constraints very frequently leads to them accidentally discovering "loopholes" that would never have occurred to a human (as with "box surfing" here). The AI doesn't know it's "cheating"; its behavior may be emergently complex, but its model of our desires is only what we tell it. A naive and unlikely example would be telling an AI to maximize human happiness and prevent human harm, so it immobilizes everyone and sticks wires into their pleasure centers. Everyone is as happy as it is possible for a human to be, and no one is doing anything remotely dangerous! The actual dangers will be stranger and harder to predict. I'm not saying we can't find a way to make strong AI safe. I'm just saying that it's a much trickier task than you imply. https://www.wired.com/story/when-bots-teach-themselves-to-cheat/ https://www.wired.com/story/when-bots-teach-themselves-to-ch... https://vkrakovna.wordpress.com/2018/04/02/specification-gaming-examples-in-ai/ https://vkrakovna.wordpress.com/2018/04/02/specification-gam...
visarga 7y ago
> The next step to make this even cooler would be to use physical, robotic agents learning to overcome challenges in real meatspace! That's one of the main challenges - how to learn safely and with fewer than millions of trials, so it can be feasible to do in the real world.
lopmotr 7y ago
> real meatspace! I'm doing something like this as a hobby but only single agent. The input is camera images and reward is based on a stopped/moving flag determined by changes between successive images as well as favoring going forward over turning. So far, it can learn to avoid crashing into walls, which is about all I'd expect. Trying to find good automated rewards without building too much special hardware is difficult. It's a vanilla DQN.

bryanrasmussen 7y ago

hmm, yes in the story I'm envisioning the AIs don't wipe out humanity because they have achieved sentience, but just because it turns out killing all humans is an optimizing component of solving some other problem.

ismail 7y ago
Asimov 3 rules as the final policy when making decisions should sort this problem out. This assumes that the rules cannot be changed by the AI.
- mithr 7y ago
  > This assumes that the rules cannot be changed by the AI. And sidesteps the fact that many of Asimov's stories were precisely about robots finding ways around these rules :)
- Falling3 7y ago
  I've always been very incredulous that there would be any possibility of taking something sufficiently complex to be considered an AGI and hard-coding anything like the 3 rules into it.
  - smogcutter 7y ago
    By the same token, I’m extremely suspicious of the idea that such a sufficiently complex AGI could also be dumb enough to optimize for paper clip production at the expense of all life on earth (or w/e example).
    - ludwigschubert 7y ago
      ...and many would say that’s because us humans are bad at imagining optimizing agents without anthropomorphizing them. This is a reasonable, even typical suspicion that many people share! The best explanation I know of why it’s unfortunately wrong is by Robert Miles in a video, but if you prefer a more thorough treatment, you could also read about “instrumental convergence” directly. If you find a flaw in this idea, I’d be interested to hear about it! :) Robert Miles’ video: https://youtu.be/ZeecOKBus3Q https://youtu.be/ZeecOKBus3Q Instrumental Convergence: https://arbital.com/p/instrumental_convergence/ https://arbital.com/p/instrumental_convergence/ Now afaik nothing in this argument says that we can’t find a way to control this in a more complex formalism-but we clearly haven’t done so yet.
      - smogcutter 7y ago
        Sorry, just saw this. I think it’s his assumption that an AGI will act strictly as an agent that’s flawed. It requires imagining an agent that can make inferences from context, evaluate new and unfamiliar information, form original plans, execute them with all the complexity implied by interaction with the real world, reprogram itself, essentially do anything... except evaluate its own terminal goal. That’s written in stone, gotta make more paperclips. The argument assumes almost unlimited power and potential on the one hand, and bizarre, arbitrary constraints on the other. If you assume an AGI is incapable of asking “why” about its terminal goal, you have to assume it’s incapable of asking “why” in any context. Miles’ AGI has no power of metacognition, but is still somehow able to reprogram itself. This really isn’t compatible with “general intelligence” or the powers that get ascribed to imaginary AGIs. I’m certainly no expert, but I expect there will turn out to be something like the idea of Turing-completeness for AI. Just like any general computing machine is a computer, any true AGI will be sapient. You can’t just arbitrarily pluck a part out, like “it can’t reason about its objective”, and expect it to still function as an AGI, just like you can’t say “it’s Turing complete, except it can’t do any kind of conditional branching.” EDIT better example: “it’s Turing complete, but it can’t do bubble sort.” This intuition may be wrong, but it’s just as much as assumption as Miles’ argument. I’m also not ascribing morality to it: we have our share of psychopaths, and intelligence doesn’t imply empathy. AGI may very well be dangerous, just probably not the “mindlessly make paperclips” kind.
CodeGlitch 7y ago
I think we humans have already solved this problem you describe... we call them laws. We use these laws to prevent people doing bad things, and I see no reason why they can't be described to an AI to drive its behavior to one that isn't going to end humanity . for the most part. fingers crossed.
- PhasmaFelis 7y ago
  I think you're misunderstanding the problem. Expressing complex rules in a machine-readable format is the least of the issues here. The main problem is that training AIs to optimize certain behaviors within constraints very frequently leads to them accidentally discovering "loopholes" that would never have occurred to a human (as with "box surfing" here). The AI doesn't know it's "cheating"; its behavior may be emergently complex, but its model of our desires is only what we tell it. A naive and unlikely example would be telling an AI to maximize human happiness and prevent human harm, so it immobilizes everyone and sticks wires into their pleasure centers. Everyone is as happy as it is possible for a human to be, and no one is doing anything remotely dangerous! The actual dangers will be stranger and harder to predict. I'm not saying we can't find a way to make strong AI safe. I'm just saying that it's a much trickier task than you imply. https://www.wired.com/story/when-bots-teach-themselves-to-cheat/ https://www.wired.com/story/when-bots-teach-themselves-to-ch... https://vkrakovna.wordpress.com/2018/04/02/specification-gaming-examples-in-ai/ https://vkrakovna.wordpress.com/2018/04/02/specification-gam...

visarga 7y ago

> The next step to make this even cooler would be to use physical, robotic agents learning to overcome challenges in real meatspace! That's one of the main challenges - how to learn safely and with fewer than millions of trials, so it can be feasible to do in the real world.

lopmotr 7y ago

> real meatspace! I'm doing something like this as a hobby but only single agent. The input is camera images and reward is based on a stopped/moving flag determined by changes between successive images as well as favoring going forward over turning. So far, it can learn to avoid crashing into walls, which is about all I'd expect. Trying to find good automated rewards without building too much special hardware is difficult. It's a vanilla DQN.