5 ms·

> 2. There is no fire alarm for AGI[1]. Is this thing the key to really dangerous AI? Will the next thing be? No one knows. It's better to be conservative. Thi

> 2. There is no fire alarm for AGI[1]. Is this thing the key to really dangerous AI? Will the next thing be? No one knows. It's better to be conservative. This is certainly a point worth discussing, but it's odd for OpenAI to suddenly stand on principle about this specific result. OpenAI was founded on the idea that public access to AI tools and research would improve and democratize them. Their website and founders make statements like "Are we really willing to let our society be infiltrated by autonomous software and hardware agents whose details of operation are known only to a select few? Of course not." and "...empower as many people as possible to have AI. If everyone has AI powers, then there's not any one person or a small set of individuals who can have AI superpower." Meanwhile, their view on AGI risk seems to be that more access will improve scrutiny and oversight. Since day one, they've taken flak from people like Nick Bostrom and Scott Alexander (https://futureoflife.org/2015/12/17/should-ai-be-open https://futureoflife.org/2015/12/17/should-ai-be-open) who argued that their stance amounted to publicly sharing untested, hazardous tools. And now they balked because... they built an essay generator that's not quite good enough to compete with cheap troll farms that governments can already afford? I don't think they're necessarily wrong to not share this, and I know Sam Altman said a while back that they might not release all their source code. But I don't think this should be looked at in isolation. If the people going to bat for democratized access to AI suddenly decided that the public can't be trusted with access to AI tools, that's worth talking about.

fossuser 8y ago
I think they did change their mind on this after considering the AGI risk more and getting a better understanding of it. I'm not sure where I read a discussion about this, but I do think it was something they've talked about before.
- goatlover 8y ago
  Is OpenAI anywhere near AGI such that it would warrant such concerns?
  - fossuser 8y ago
    It's hard to know what near is and what the timeline is. This article digs into some historical evidence of this: https://www.lesswrong.com/posts/BEtzRE2M5m9YEAQpX/there-s-no-fire-alarm-for-artificial-general-intelligence https://www.lesswrong.com/posts/BEtzRE2M5m9YEAQpX/there-s-no... From the post: "Two: History shows that for the general public, and even for scientists not in a key inner circle, and even for scientists in that key circle, it is very often the case that key technological developments still seem decades away, five years before they show up. In 1901, two years before helping build the first heavier-than-air flyer, Wilbur Wright told his brother that powered flight was fifty years away. In 1939, three years before he personally oversaw the first critical chain reaction in a pile of uranium bricks, Enrico Fermi voiced 90% confidence that it was impossible to use uranium to sustain a fission chain reaction. I believe Fermi also said a year after that, aka two years before the denouement, that if net power from fission was even possible (as he then granted some greater plausibility) then it would be fifty years off; but for this I neglected to keep the citation. And of course if you're not the Wright Brothers or Enrico Fermi, you will be even more surprised. Most of the world learned that atomic weapons were now a thing when they woke up to the headlines about Hiroshima. There were esteemed intellectuals saying four years after the Wright Flyer that heavier-than-air flight was impossible, because knowledge propagated more slowly back then." Maybe it's hundreds of years out, but maybe it's not. Since it's hard to know probably better to try and work on the safety aspects now.
    - dannyw 8y ago
      Of course, I'm sure you can find plenty of examples where things that are decades away, are described as decades away.
      - fossuser 8y ago
        Sure - this isn’t to imply that that can’t happen or even that that isn’t the more common case. It’s only to say that just because something may seem far away it may not be - even if you’re the person that will invent it only two years later (and therefore are probably in the best position to know). Given the high stakes of unsafe AGI and this uncertainty it’s probably worth some people working on goal alignment. This is somewhat unrelated to the recent release though.
hyperpallium 8y ago
I, for one, welcome our new OpenAI overlords.
deleted 8y ago
[deleted]

fossuser 8y ago

I think they did change their mind on this after considering the AGI risk more and getting a better understanding of it. I'm not sure where I read a discussion about this, but I do think it was something they've talked about before.

goatlover 8y ago
Is OpenAI anywhere near AGI such that it would warrant such concerns?
- fossuser 8y ago
  It's hard to know what near is and what the timeline is. This article digs into some historical evidence of this: https://www.lesswrong.com/posts/BEtzRE2M5m9YEAQpX/there-s-no-fire-alarm-for-artificial-general-intelligence https://www.lesswrong.com/posts/BEtzRE2M5m9YEAQpX/there-s-no... From the post: "Two: History shows that for the general public, and even for scientists not in a key inner circle, and even for scientists in that key circle, it is very often the case that key technological developments still seem decades away, five years before they show up. In 1901, two years before helping build the first heavier-than-air flyer, Wilbur Wright told his brother that powered flight was fifty years away. In 1939, three years before he personally oversaw the first critical chain reaction in a pile of uranium bricks, Enrico Fermi voiced 90% confidence that it was impossible to use uranium to sustain a fission chain reaction. I believe Fermi also said a year after that, aka two years before the denouement, that if net power from fission was even possible (as he then granted some greater plausibility) then it would be fifty years off; but for this I neglected to keep the citation. And of course if you're not the Wright Brothers or Enrico Fermi, you will be even more surprised. Most of the world learned that atomic weapons were now a thing when they woke up to the headlines about Hiroshima. There were esteemed intellectuals saying four years after the Wright Flyer that heavier-than-air flight was impossible, because knowledge propagated more slowly back then." Maybe it's hundreds of years out, but maybe it's not. Since it's hard to know probably better to try and work on the safety aspects now.
  - dannyw 8y ago
    Of course, I'm sure you can find plenty of examples where things that are decades away, are described as decades away.
    - fossuser 8y ago
      Sure - this isn’t to imply that that can’t happen or even that that isn’t the more common case. It’s only to say that just because something may seem far away it may not be - even if you’re the person that will invent it only two years later (and therefore are probably in the best position to know). Given the high stakes of unsafe AGI and this uncertainty it’s probably worth some people working on goal alignment. This is somewhat unrelated to the recent release though.

hyperpallium 8y ago

I, for one, welcome our new OpenAI overlords.

deleted 8y ago

[deleted]