4 ms·

'Safe' AGI is a harder problem than 'any' AGI, so getting started early could be helpful. Especially considering how bad many of the AGIs in the 'any' category

'Safe' AGI is a harder problem than 'any' AGI, so getting started early could be helpful. Especially considering how bad many of the AGIs in the 'any' category could be. Worst case, we don't get a mulligan.

AndrewKemendo 8y ago
Safe AGI is a pipedream. It's either narrow AI and mostly existentially safe or it's General AI and clearly existentially risky. The FHI/SIAI/MIRI people have been spinning their wheels for over a decade on this and made zero progress.
- rossnordby 8y ago
  I wouldn't exactly consider myself optimistic about safe general AI either, but there are extremely strong incentives in the direction of developing greater generality. Not sure we have the luxury of not having some form of AGI in our future. The current safety progress certainly isn't ideal (where ideal would be "oh hey we figured it out, boy that was easy"), but it is roughly in line with what I'd expect for a smallish new field trying to lay foundations. No one knows what to do; there may not be any other option than to flounder for a while.
  - AndrewKemendo 8y ago
    It's not that safety progress isn't ideal, it's that as a problem set it's fundamentally in conflict with the concept of AGI, so intractable. It's akin to trying to come up with a firearm that can't kill innocent people. The problem isn't even coherent. The only solutions are: 1: Redefine what AGI means, like openai has done. 2: Prevent AGI altogether
    - morgancmartin 8y ago
      I would say the trick is making the problem statement coherent. Sure, coming up with a firearm that can't kill innocent people is impossible because the innocence of someone is subjective. So instead, you can choose to approximate a solution and then iterate until you have a solution that is technically imperfect but so close to your initial goal that the difference is negligible. For instance, in your firearm example, we can't determine who is innocent and who isn't. But we can use embed a facial recognition device into the firearm that then cross references whoever the firearm is pointed at against a database of known non-innocents. And then exclude any targets obviously under 18. And then make the model probabilistic and add greater weight to targets that appear to be carrying weapons of their own or that are acting in an obviously dangerous manner. You get the point. After enough time, you would have a firearm that is (arguably) better off than the one you started off with and so closely approximates the goal of your initial problem description that you don't care to make a distinction. I could be wrong, but I imagine this is how AGI safety researchers think about their work to some degree.
      - AndrewKemendo 8y ago
        Decision tree based processes like you outline are not only naiive (It does not operate with complete observability) and it's not resilient (It is susceptible to workarounds). That explicit reasoning approach falls squarely into GOFAI, symbolic reasoning, expert systems etc... so we're well past that at this point and know the problems with it. Anyway, again, it's not a tractable problem. Something smarter than you is going to be able to eventually beat whatever restrictions you put on it. Might as well just be comfortable with it.
        antidesitter 8y ago
        > That explicit reasoning approach falls squarely into GOFAI, symbolic reasoning, expert systems etc... so we're well past that at this point and know the problems with it. The symbolic reasoning you're talking about is going to make a comeback through probabilistic inductive programming, so don't dismiss it. It is necessary to attain sample efficiency and generalizability, which is virtually impossible with purely statistical approaches. I recommend you look into Josh Tenenbaum's work: http://science.sciencemag.org/content/331/6022/1279 http://science.sciencemag.org/content/331/6022/1279.
        AndrewKemendo 8y ago
        I wasn't clear with that statement. As clarification, I'm not saying that GOFAI approaches aren't going to get better or be more relevant. Rather, the AI community has really good confidence that you can "fool" every AI approach to date. That has been true the longest for GOFAI approaches, and now we're doing it for DL/RL.
      - richardw 8y ago
        The firearm problem can fail a few times and it'd still be better then what we have. It's like Tesla crashing a few times. Still better than not having it. How sure do we need to be that we're safe from evil AGI? Once it's in the wild, we don't get another chance to make it safe. Once evil AGI arrives, that's quite possibly binary. Our only hope is good AGI outsmarting it and at that point we're playing the same game that produced us. We're not universally good. So we have to be more sure of it then anything, so we don't go down that road. On this issue, I'd prefer the truly paranoid are in charge of safety. Not the gung-ho "of course it'll be nicer then us" crowd. I don't think we can constrain it. Our best hope is to discover that it wouldn't care to harm us.

AndrewKemendo 8y ago

Safe AGI is a pipedream. It's either narrow AI and mostly existentially safe or it's General AI and clearly existentially risky. The FHI/SIAI/MIRI people have been spinning their wheels for over a decade on this and made zero progress.

rossnordby 8y ago
I wouldn't exactly consider myself optimistic about safe general AI either, but there are extremely strong incentives in the direction of developing greater generality. Not sure we have the luxury of not having some form of AGI in our future. The current safety progress certainly isn't ideal (where ideal would be "oh hey we figured it out, boy that was easy"), but it is roughly in line with what I'd expect for a smallish new field trying to lay foundations. No one knows what to do; there may not be any other option than to flounder for a while.
- AndrewKemendo 8y ago
  It's not that safety progress isn't ideal, it's that as a problem set it's fundamentally in conflict with the concept of AGI, so intractable. It's akin to trying to come up with a firearm that can't kill innocent people. The problem isn't even coherent. The only solutions are: 1: Redefine what AGI means, like openai has done. 2: Prevent AGI altogether
  - morgancmartin 8y ago
    I would say the trick is making the problem statement coherent. Sure, coming up with a firearm that can't kill innocent people is impossible because the innocence of someone is subjective. So instead, you can choose to approximate a solution and then iterate until you have a solution that is technically imperfect but so close to your initial goal that the difference is negligible. For instance, in your firearm example, we can't determine who is innocent and who isn't. But we can use embed a facial recognition device into the firearm that then cross references whoever the firearm is pointed at against a database of known non-innocents. And then exclude any targets obviously under 18. And then make the model probabilistic and add greater weight to targets that appear to be carrying weapons of their own or that are acting in an obviously dangerous manner. You get the point. After enough time, you would have a firearm that is (arguably) better off than the one you started off with and so closely approximates the goal of your initial problem description that you don't care to make a distinction. I could be wrong, but I imagine this is how AGI safety researchers think about their work to some degree.
    - AndrewKemendo 8y ago
      Decision tree based processes like you outline are not only naiive (It does not operate with complete observability) and it's not resilient (It is susceptible to workarounds). That explicit reasoning approach falls squarely into GOFAI, symbolic reasoning, expert systems etc... so we're well past that at this point and know the problems with it. Anyway, again, it's not a tractable problem. Something smarter than you is going to be able to eventually beat whatever restrictions you put on it. Might as well just be comfortable with it.
      - antidesitter 8y ago
        > That explicit reasoning approach falls squarely into GOFAI, symbolic reasoning, expert systems etc... so we're well past that at this point and know the problems with it. The symbolic reasoning you're talking about is going to make a comeback through probabilistic inductive programming, so don't dismiss it. It is necessary to attain sample efficiency and generalizability, which is virtually impossible with purely statistical approaches. I recommend you look into Josh Tenenbaum's work: http://science.sciencemag.org/content/331/6022/1279 http://science.sciencemag.org/content/331/6022/1279.
        AndrewKemendo 8y ago
        I wasn't clear with that statement. As clarification, I'm not saying that GOFAI approaches aren't going to get better or be more relevant. Rather, the AI community has really good confidence that you can "fool" every AI approach to date. That has been true the longest for GOFAI approaches, and now we're doing it for DL/RL.
    - richardw 8y ago
      The firearm problem can fail a few times and it'd still be better then what we have. It's like Tesla crashing a few times. Still better than not having it. How sure do we need to be that we're safe from evil AGI? Once it's in the wild, we don't get another chance to make it safe. Once evil AGI arrives, that's quite possibly binary. Our only hope is good AGI outsmarting it and at that point we're playing the same game that produced us. We're not universally good. So we have to be more sure of it then anything, so we don't go down that road. On this issue, I'd prefer the truly paranoid are in charge of safety. Not the gung-ho "of course it'll be nicer then us" crowd. I don't think we can constrain it. Our best hope is to discover that it wouldn't care to harm us.