4 ms·

I'm curious about the emphasis on safety. I don't think anyone claims they are closer to achieving AGI. And yet there are already jobs for AGI safety ? :)

bitL 8y ago
Replace it with older term "censorship" perhaps. Facebook is hiring into its community safety teams too, apply now ;-)
rossnordby 8y ago
'Safe' AGI is a harder problem than 'any' AGI, so getting started early could be helpful. Especially considering how bad many of the AGIs in the 'any' category could be. Worst case, we don't get a mulligan.
- AndrewKemendo 8y ago
  Safe AGI is a pipedream. It's either narrow AI and mostly existentially safe or it's General AI and clearly existentially risky. The FHI/SIAI/MIRI people have been spinning their wheels for over a decade on this and made zero progress.
  - rossnordby 8y ago
    I wouldn't exactly consider myself optimistic about safe general AI either, but there are extremely strong incentives in the direction of developing greater generality. Not sure we have the luxury of not having some form of AGI in our future. The current safety progress certainly isn't ideal (where ideal would be "oh hey we figured it out, boy that was easy"), but it is roughly in line with what I'd expect for a smallish new field trying to lay foundations. No one knows what to do; there may not be any other option than to flounder for a while.
    - AndrewKemendo 8y ago
      It's not that safety progress isn't ideal, it's that as a problem set it's fundamentally in conflict with the concept of AGI, so intractable. It's akin to trying to come up with a firearm that can't kill innocent people. The problem isn't even coherent. The only solutions are: 1: Redefine what AGI means, like openai has done. 2: Prevent AGI altogether
      - morgancmartin 8y ago
        I would say the trick is making the problem statement coherent. Sure, coming up with a firearm that can't kill innocent people is impossible because the innocence of someone is subjective. So instead, you can choose to approximate a solution and then iterate until you have a solution that is technically imperfect but so close to your initial goal that the difference is negligible. For instance, in your firearm example, we can't determine who is innocent and who isn't. But we can use embed a facial recognition device into the firearm that then cross references whoever the firearm is pointed at against a database of known non-innocents. And then exclude any targets obviously under 18. And then make the model probabilistic and add greater weight to targets that appear to be carrying weapons of their own or that are acting in an obviously dangerous manner. You get the point. After enough time, you would have a firearm that is (arguably) better off than the one you started off with and so closely approximates the goal of your initial problem description that you don't care to make a distinction. I could be wrong, but I imagine this is how AGI safety researchers think about their work to some degree.
        AndrewKemendo 8y ago
        Decision tree based processes like you outline are not only naiive (It does not operate with complete observability) and it's not resilient (It is susceptible to workarounds). That explicit reasoning approach falls squarely into GOFAI, symbolic reasoning, expert systems etc... so we're well past that at this point and know the problems with it. Anyway, again, it's not a tractable problem. Something smarter than you is going to be able to eventually beat whatever restrictions you put on it. Might as well just be comfortable with it.
currymj 8y ago
This website is made by and somewhat for a subculture of people very worried about safe AGI. The idea is it's a career guide for people who want to know how to have a career that will maximize some definition of total utility (in their view, this is the morally best career). They see preventing unfriendly AGI as a very important goal in a utilitarian sense.

bitL 8y ago

Replace it with older term "censorship" perhaps. Facebook is hiring into its community safety teams too, apply now ;-)

rossnordby 8y ago

'Safe' AGI is a harder problem than 'any' AGI, so getting started early could be helpful. Especially considering how bad many of the AGIs in the 'any' category could be. Worst case, we don't get a mulligan.

AndrewKemendo 8y ago
Safe AGI is a pipedream. It's either narrow AI and mostly existentially safe or it's General AI and clearly existentially risky. The FHI/SIAI/MIRI people have been spinning their wheels for over a decade on this and made zero progress.
- rossnordby 8y ago
  I wouldn't exactly consider myself optimistic about safe general AI either, but there are extremely strong incentives in the direction of developing greater generality. Not sure we have the luxury of not having some form of AGI in our future. The current safety progress certainly isn't ideal (where ideal would be "oh hey we figured it out, boy that was easy"), but it is roughly in line with what I'd expect for a smallish new field trying to lay foundations. No one knows what to do; there may not be any other option than to flounder for a while.
  - AndrewKemendo 8y ago
    It's not that safety progress isn't ideal, it's that as a problem set it's fundamentally in conflict with the concept of AGI, so intractable. It's akin to trying to come up with a firearm that can't kill innocent people. The problem isn't even coherent. The only solutions are: 1: Redefine what AGI means, like openai has done. 2: Prevent AGI altogether
    - morgancmartin 8y ago
      I would say the trick is making the problem statement coherent. Sure, coming up with a firearm that can't kill innocent people is impossible because the innocence of someone is subjective. So instead, you can choose to approximate a solution and then iterate until you have a solution that is technically imperfect but so close to your initial goal that the difference is negligible. For instance, in your firearm example, we can't determine who is innocent and who isn't. But we can use embed a facial recognition device into the firearm that then cross references whoever the firearm is pointed at against a database of known non-innocents. And then exclude any targets obviously under 18. And then make the model probabilistic and add greater weight to targets that appear to be carrying weapons of their own or that are acting in an obviously dangerous manner. You get the point. After enough time, you would have a firearm that is (arguably) better off than the one you started off with and so closely approximates the goal of your initial problem description that you don't care to make a distinction. I could be wrong, but I imagine this is how AGI safety researchers think about their work to some degree.
      - AndrewKemendo 8y ago
        Decision tree based processes like you outline are not only naiive (It does not operate with complete observability) and it's not resilient (It is susceptible to workarounds). That explicit reasoning approach falls squarely into GOFAI, symbolic reasoning, expert systems etc... so we're well past that at this point and know the problems with it. Anyway, again, it's not a tractable problem. Something smarter than you is going to be able to eventually beat whatever restrictions you put on it. Might as well just be comfortable with it.

currymj 8y ago

This website is made by and somewhat for a subculture of people very worried about safe AGI. The idea is it's a career guide for people who want to know how to have a career that will maximize some definition of total utility (in their view, this is the morally best career). They see preventing unfriendly AGI as a very important goal in a utilitarian sense.