3 ms·

Thanks for sharing. One thing I was interested to read but didn't find in your post is: does everyone believe in the vision that the leadership has shared publ

by imiric 1y ago

Thanks for sharing. One thing I was interested to read but didn't find in your post is: does everyone believe in the vision that the leadership has shared publicly, e.g. [1]? Is there some skepticism that the current path leads to AGI, or has everyone drunk the Kool-Aid? If there is some dissent, how is it handled internally? [1]: https://blog.samaltman.com/the-gentle-singularity https://blog.samaltman.com/the-gentle-singularity

fragmede 1y ago
Externally there's no rigorous definition as to what constitutes AGI, so I'd guess internally it's not one monolithic thing they're targeting either. You'd need everyone to take a class about the nature of intelligence first, and all the different kinds of it just to begin with. There's undoubtedly dissent internally as to the best way to achieve chosen milestones on the way there, as well as disagreement that those are the right milestones to begin with. Think tactical disagreement, not strategic. If you didn't think that AGI were ever possible with LLMs, would you even be there to begin with?
- imiric 1y ago
  Well, Sam Altman has a clear definition of ASI, and AGI is something they've been thinking about for a long time, so presumably they must have some accepted definition of it. My question was whether everyone believes this vision that ASI is "close", and more broadly whether this path leads to AGI. > If you didn't think that AGI were ever possible with LLMs, would you even be there to begin with? People can have all sorts of reasons for working with a company. They might want to work on cutting-edge tech with smart people and infinite resources, for investment or prestige, but not necessarily buy into the overarching vision. I'm just wondering whether such a profile exists within OpenAI, and if so, how it is handled.
tedsanders 1y ago
Not the author, but I work at OpenAI. There are wide variety of viewpoints and it's fine for employees to disagree on timelines and impact. I myself published a 100-page paper on why I think transformative AGI by 2043 is quite unlikely (https://arxiv.org/abs/2306.02519 https://arxiv.org/abs/2306.02519). From informal discussion, I think the vast majority of employees don't think that we're mere years from a post-scarcity utopia where we can drink mai tais on the beach all day. But there is a lot of optimism about the rapid progress in AI, and I do think that it's harder to forecast the path of a technology that has the potential to improve itself. So much depends on your definition of AGI. In a sense, GPT-4 is already AGI in the literal sense that it's an artificial intelligence with some generality. But in the sense of automating the economy, it's of course not close.
- criddell 1y ago
  > depends on your definition of AGI What definition of AGI is used at OpenAI? My definition: AGI will be here when you can put it in a robot body in the real word and interact with it like you would a person. Ask it to drive your car or fold your laundry or make a mai tai and if it doesn’t know how to do that, you show it, and then it can.
  - tedsanders 1y ago
    In the OpenAI charter, it's "highly autonomous systems that outperform humans at most economically valuable work." https://openai.com/charter/ https://openai.com/charter/
    - criddell 1y ago
      Huh. That feels like kind of a weak definition. That makes me wonder what kinds of work aren’t economically valuable? Would that be services generally provided by government?
      - tedsanders 1y ago
        Maybe I'm biased, but I actually think it's a pretty good definition, as definitions go. All of our narrow measures of human intelligence that we might be tempted to use - win at games, solve math problems, ace academic tests, dominate at programming competitions - are revealed as woefully insufficient as soon as an AI beats them but fails to generalize far beyond. But if you have an AI that can generate lots of revenue doing a wide variety of real work, then you've probably built something smart. Diverse revenue is a great metric.
        criddell 1y ago
        I also find it interesting that the definition always includes the "outperforms humans" qualifier. Maybe our first AGIs will underperform humans. Imagine I built a robot dog that behaved just like a biological dog. It bonds with people, can be trained, shows emotion, communicates, likes to play, likes to work, solves problems, understands social cues, and is loyal. IMHO, that would qualify as an AGI even though it isn't writing essays or producing business plans.

by imiric 1y ago

fragmede 1y ago

Externally there's no rigorous definition as to what constitutes AGI, so I'd guess internally it's not one monolithic thing they're targeting either. You'd need everyone to take a class about the nature of intelligence first, and all the different kinds of it just to begin with. There's undoubtedly dissent internally as to the best way to achieve chosen milestones on the way there, as well as disagreement that those are the right milestones to begin with. Think tactical disagreement, not strategic. If you didn't think that AGI were ever possible with LLMs, would you even be there to begin with?

imiric 1y ago
Well, Sam Altman has a clear definition of ASI, and AGI is something they've been thinking about for a long time, so presumably they must have some accepted definition of it. My question was whether everyone believes this vision that ASI is "close", and more broadly whether this path leads to AGI. > If you didn't think that AGI were ever possible with LLMs, would you even be there to begin with? People can have all sorts of reasons for working with a company. They might want to work on cutting-edge tech with smart people and infinite resources, for investment or prestige, but not necessarily buy into the overarching vision. I'm just wondering whether such a profile exists within OpenAI, and if so, how it is handled.

tedsanders 1y ago

Not the author, but I work at OpenAI. There are wide variety of viewpoints and it's fine for employees to disagree on timelines and impact. I myself published a 100-page paper on why I think transformative AGI by 2043 is quite unlikely (https://arxiv.org/abs/2306.02519 https://arxiv.org/abs/2306.02519). From informal discussion, I think the vast majority of employees don't think that we're mere years from a post-scarcity utopia where we can drink mai tais on the beach all day. But there is a lot of optimism about the rapid progress in AI, and I do think that it's harder to forecast the path of a technology that has the potential to improve itself. So much depends on your definition of AGI. In a sense, GPT-4 is already AGI in the literal sense that it's an artificial intelligence with some generality. But in the sense of automating the economy, it's of course not close.

criddell 1y ago
> depends on your definition of AGI What definition of AGI is used at OpenAI? My definition: AGI will be here when you can put it in a robot body in the real word and interact with it like you would a person. Ask it to drive your car or fold your laundry or make a mai tai and if it doesn’t know how to do that, you show it, and then it can.
- tedsanders 1y ago
  In the OpenAI charter, it's "highly autonomous systems that outperform humans at most economically valuable work." https://openai.com/charter/ https://openai.com/charter/
  - criddell 1y ago
    Huh. That feels like kind of a weak definition. That makes me wonder what kinds of work aren’t economically valuable? Would that be services generally provided by government?
    - tedsanders 1y ago
      Maybe I'm biased, but I actually think it's a pretty good definition, as definitions go. All of our narrow measures of human intelligence that we might be tempted to use - win at games, solve math problems, ace academic tests, dominate at programming competitions - are revealed as woefully insufficient as soon as an AI beats them but fails to generalize far beyond. But if you have an AI that can generate lots of revenue doing a wide variety of real work, then you've probably built something smart. Diverse revenue is a great metric.
      - criddell 1y ago
        I also find it interesting that the definition always includes the "outperforms humans" qualifier. Maybe our first AGIs will underperform humans. Imagine I built a robot dog that behaved just like a biological dog. It bonds with people, can be trained, shows emotion, communicates, likes to play, likes to work, solves problems, understands social cues, and is loyal. IMHO, that would qualify as an AGI even though it isn't writing essays or producing business plans.