3 ms·

> The fact that scaling continues to work has significant implications for the timelines of AGI development. The scaling hypothesis is the idea that we may have

> The fact that scaling continues to work has significant implications for the timelines of AGI development. The scaling hypothesis is the idea that we may have most of the pieces in place needed to build AGI and that most of the remaining work will be taking existing methods and scaling them up. Is this a pretty consensual viewpoint? Or is it hype from OpenAI?

meroes 3y ago
Within comp sci? Not sure. Within academia writ large? No way. But hey I look to biologists who put biological immortality > 100-200 years out (I've never believed Aubrey de Grey and similars during the 20 years he's been in headlines) and see AGI as in the domain of biology/psychology. We've implicated the brain, that's about it.
- sebzim4500 3y ago
  It seems absurd to me that someone would try to make a prediction about the state of technology in 100 years. If you asked people in 1920 to make a list of predictions about what we would/wouldn't be able to do today would they have done better than chance? I guess our understanding of the laws of physics haven't changed that much. So someone predicting that free energy will be impossible is likely correct.
  - warkdarrior 3y ago
    Solar power is mostly free energy, at least for the next 5B years.
    - twelve40 3y ago
      uh... also "free" is hydro and wind which people have been using for millenniums
wongarsu 3y ago
Taking existing methods and scaling them up leads to better LLMs. That's a major part of what OpenAI is doing. Now the scaling hypothesis is basically just that sophisticated behavior emerges by itself as neural networks get bigger and get trained on more data with more complex problems. LLMs certainly show that so far, but to jump to an AGI I think we need to fix various shortcomings. It's certainly possible that a future version of an LLM will be an AGI, but imho that will also be thanks to current and future improvements like adding an internal dialogue or medium-term memory. Or maybe LLMs just can't scale that far, because experiencing the world through text is insufficient.
- moffkalast 3y ago
  Yeah at this point nobody has the authority to say how far the architecture might go. It looks promising, OpenAI has all the reason to hype it as much as possible and it certainly seems plausible, but it could just as likely hit a practical ceiling of some sort.

meroes 3y ago

Within comp sci? Not sure. Within academia writ large? No way. But hey I look to biologists who put biological immortality > 100-200 years out (I've never believed Aubrey de Grey and similars during the 20 years he's been in headlines) and see AGI as in the domain of biology/psychology. We've implicated the brain, that's about it.

sebzim4500 3y ago
It seems absurd to me that someone would try to make a prediction about the state of technology in 100 years. If you asked people in 1920 to make a list of predictions about what we would/wouldn't be able to do today would they have done better than chance? I guess our understanding of the laws of physics haven't changed that much. So someone predicting that free energy will be impossible is likely correct.
- warkdarrior 3y ago
  Solar power is mostly free energy, at least for the next 5B years.
  - twelve40 3y ago
    uh... also "free" is hydro and wind which people have been using for millenniums

wongarsu 3y ago

Taking existing methods and scaling them up leads to better LLMs. That's a major part of what OpenAI is doing. Now the scaling hypothesis is basically just that sophisticated behavior emerges by itself as neural networks get bigger and get trained on more data with more complex problems. LLMs certainly show that so far, but to jump to an AGI I think we need to fix various shortcomings. It's certainly possible that a future version of an LLM will be an AGI, but imho that will also be thanks to current and future improvements like adding an internal dialogue or medium-term memory. Or maybe LLMs just can't scale that far, because experiencing the world through text is insufficient.

moffkalast 3y ago
Yeah at this point nobody has the authority to say how far the architecture might go. It looks promising, OpenAI has all the reason to hype it as much as possible and it certainly seems plausible, but it could just as likely hit a practical ceiling of some sort.