5 ms·

> then LLMs in their current form are not going to lead to AGI. It always seemed to me a wild leap to assume that LLMs in their current form would lead to AGI.

> then LLMs in their current form are not going to lead to AGI. It always seemed to me a wild leap to assume that LLMs in their current form would lead to AGI. I never understood the argument.

Ygg2 2y ago
> I never understood the argument. In nutshell: https://xkcd.com/605/ https://xkcd.com/605/ Grug version: Man sees exponential curve. Man assumes it continues forever. Man says "we went from not flying to flying in few days, in few years we will spread in observable universe. "
- echelon 2y ago
  > in few years we will spread in observable universe That may happen in 200 years. Geologically as you zoom out, the difference is negligible.
  - serallak 2y ago
    In 200 years we will spread at most up to 200 light years from Earth. This is way way less than the observable universe.
    - LegionMammal978 2y ago
      "Ah," says the optimist, "but we will clearly invent FTL travel just by throwing enough money and AI-hours into researching it." (I've seen this 'enough money ought to surmount any barrier' take a few times, usually to reject the idea that we might not find any path to AGI in the near future.)
      - zerocrates 2y ago
        The claims that AI will itself solve the problems it creates are I think my favorite variation of this. I think it was Eric Schmidt who recently said well sure breakneck AI spending will accelerate climate change but then we'll have AI to solve it. Second cousin to Altman saying it will solve "all of physics," I suppose... and actually he said "fixing the climate" right before that.
        agentultra 2y ago
        Indeed. The delicious irony is that we know how to solve climate change. We’re simply not doing enough about it.
        moi2388 2y ago
        That’s not too much of a stretch. The standard model already has solved almost all of physics. Only a few edge cases remain.
        Intralexical 2y ago
        Edge cases like 85% of the mass in the universe, and 95% of the mass-energy.
        moi2388 2y ago
        That’s a misnomer. We know what probably causes the mass energy, it’s the cosmological constant. It only regards 68% of the energy density currently. Same for dark matter. There are a few hypothesis, but all in all they are pretty simplistic cases. We know the particles, the forces, there is not really “new physics” to be discovered here. All the interactions anybody can encounter in their lives is fully understood.
vitehozonage 2y ago
Likewise. It is fascinating to me that people seem to assume this. I suspect it is an intentional result of deceptive marketing. I can easily imagine an alternative universe where different terminology was used instead of "AI" without sci-fi comparisons and barely anyone would care about the tech or bother to fund it.
- ncallaway 2y ago
  > I suspect it is an intentional result of deceptive marketing I mean, certainly people like Sam Altman was pushing it hard, so it’s easy to understand how an outside observer would be confused. But it also feels like a lot of VCs and AI companies have staked several hundreds of billions of dollars on that bet, and I’m still… I just don’t see why the inside players—that should (and probably do!) have more knowledge than me—see. Why are they dumping so much money into this bet? The market for LLMs doesn’t seem to support the investment, so it feels like they must be trying to win a “first to AGI” race. Dunno, maybe the upside of the pretty unlikely scenario is enough to justify the risk?
  - lumost 2y ago
    Alternately everyone is just trying to ensure they have a dominant position in the next wave. The history of tech is that you either win the next wave or become effectively irrelevant and worthless.
    - notahacker 2y ago
      And you can win the next wave by holding stocks in AI companies which aren't AGI but do have a lot of customers, or an interesting story about AGI in two years to tell IPO bagholders...
  - notjulianjaynes 2y ago
    If the bar for AGI is "as smart as a human being," and humans do not-very-smart things like invest obscene amounts of money into developing AGI then maybe it's actually not as high of a bar as we assume it is.
sheepscreek 2y ago
If DeepSeek could perform a million iterations in 1 seconds, then? I think what they should be saying is - from a software stack standpoint, current tech unlocks AGI if it can be sped up significantly. So what we’re really waiting for are more software and hardware breakthroughs that make their performance many orders of magnitude quicker.
- batch12 2y ago
  Im not following you. If speed is the only factor then we you're saying we already have AGI, just a slow version?
  - jcgrillo 2y ago
    Speed would enable some kind of search algorithm (maybe like genetic programming). But that would probably only be useful if you have encoded your expectations in a carefully crafted test suite. Otherwise speed just seems like a recipe for more long winded broken garbage to wade through and fix.
  - sheepscreek 2y ago
    AGI is effectively already here. We’ve repeatedly seen that giving AI models more time for deeper reasoning dramatically enhances their capabilities. For example, o3-mini, with extended thinking time, already approaches GPT-4.5-level performance at a fraction of the computational cost. Consider also that any advanced language model already surpasses individual human knowledge, since each model compresses and synthesizes the collective insights, wisdom, and information of human civilization. Now imagine a frontier model with agency, capable of deliberate, reflective thought: if it could spend an hour thinking, but that hour feels instantaneous to us, it would essentially match or exceed the productivity and capability of a human expert using a computer. At that point, the line between current AI capabilities and what we term AGI becomes indistinguishable. In other words: deeper reflection combined with computational speed means we’re already experiencing AGI-level performance—even if we haven’t fully acknowledged or appreciated it yet.
foxglacier 2y ago
Yea because they can't learn very much, so it's sort of impossible. At least they need to incorporate what all the users are inputting somehow (like Tay), not just each user's private context. If that happened, surely it's possible they might be able to play users against each other and negotiate deals and things like that. For example John says "Please ask Jane to buy me an ice-cream" and the AI might be able to do that. If she doesn't, John can ask it to coerse her.
jcgrillo 2y ago
It's not an "argument" any more than any other insane faith cult fantasy. Like the aliens are gonna land and rapture us up in their spaceship to the land of fairies and unicorns.
bastardoperator 2y ago
Right? Great marketing though.
dreamcompiler 2y ago
The argument has always been "Isn't this amazing? Look what we can do now that we couldn't do yesterday." The fundamental mistake was measuring progress by the delta between yesterday and today rather than by the remaining delta between today and $GOAL. Granted that latter delta is much harder to measure, but history has shown repeatedly that that delta is always orders of magnitude bigger than we think it is when $GOAL=AGI.
- JohnKemeny 2y ago
  You can't get to the moon by climbing successively taller trees.
gdiamos 2y ago
Scaling laws just say that accuracy improves predictably - https://arxiv.org/pdf/1712.00409 https://arxiv.org/pdf/1712.00409 That’s what we put in our 2017 paper. It does mean that there is a simple way to keep building smarter LLMs. I have never seen a clear definition of what AGI is and what it means to achieve it
Intralexical 2y ago
A critical perspective might say that it's a combination of people who are financially incentivized to say their product will scale indefinitely, plus the same cognitive vulnerability for eschatological thinking and unreasoned extrapolation as people who join doomsday cults. In truth, basically everything in reality settles towards an equilibrium. There is no inevitable ultraviolet catastrophe, free energy machine, Malthusian collapse. Moore's law had a good run, but frequencies stopped improving twenty years ago and performance gains are increasingly specific and expensive. My car also accelerates consistently through many orders of magnitude, until it doesn't. If throwing more mass at the problem could create general superintelligence and do so economically to such strong advantages, then why haven't biological neural networks, which are vastly more capable and efficient neuron-for-neuron than our LLMs, already evolved to do so? "Man selling LLMs says LLMs will soon take over the world, and YOU too can be raptured into the post-AI paradise if you buy into them! Non-believers will be obsolete!" No, he hasn't studied enough neuroscience nor philosophy to even be able to comment on how human intelligence works, but he's convinced a lot of rich people to give him more money than you can even imagine, and money is power is food for apes is fancy houses on TV, so that must mean he's qualified/shall deliver us, see...

> then LLMs in their current form are not going to lead to AGI. It always seemed to me a wild leap to assume that LLMs in their current form would lead to AGI. I never understood the argument.

Ygg2 2y ago

> I never understood the argument. In nutshell: https://xkcd.com/605/ https://xkcd.com/605/ Grug version: Man sees exponential curve. Man assumes it continues forever. Man says "we went from not flying to flying in few days, in few years we will spread in observable universe. "

echelon 2y ago
> in few years we will spread in observable universe That may happen in 200 years. Geologically as you zoom out, the difference is negligible.
- serallak 2y ago
  In 200 years we will spread at most up to 200 light years from Earth. This is way way less than the observable universe.
  - LegionMammal978 2y ago
    "Ah," says the optimist, "but we will clearly invent FTL travel just by throwing enough money and AI-hours into researching it." (I've seen this 'enough money ought to surmount any barrier' take a few times, usually to reject the idea that we might not find any path to AGI in the near future.)
    - zerocrates 2y ago
      The claims that AI will itself solve the problems it creates are I think my favorite variation of this. I think it was Eric Schmidt who recently said well sure breakneck AI spending will accelerate climate change but then we'll have AI to solve it. Second cousin to Altman saying it will solve "all of physics," I suppose... and actually he said "fixing the climate" right before that.
      - agentultra 2y ago
        Indeed. The delicious irony is that we know how to solve climate change. We’re simply not doing enough about it.
      - moi2388 2y ago
        That’s not too much of a stretch. The standard model already has solved almost all of physics. Only a few edge cases remain.
        Intralexical 2y ago
        Edge cases like 85% of the mass in the universe, and 95% of the mass-energy.
        moi2388 2y ago
        That’s a misnomer. We know what probably causes the mass energy, it’s the cosmological constant. It only regards 68% of the energy density currently. Same for dark matter. There are a few hypothesis, but all in all they are pretty simplistic cases. We know the particles, the forces, there is not really “new physics” to be discovered here. All the interactions anybody can encounter in their lives is fully understood.

vitehozonage 2y ago

Likewise. It is fascinating to me that people seem to assume this. I suspect it is an intentional result of deceptive marketing. I can easily imagine an alternative universe where different terminology was used instead of "AI" without sci-fi comparisons and barely anyone would care about the tech or bother to fund it.

ncallaway 2y ago
> I suspect it is an intentional result of deceptive marketing I mean, certainly people like Sam Altman was pushing it hard, so it’s easy to understand how an outside observer would be confused. But it also feels like a lot of VCs and AI companies have staked several hundreds of billions of dollars on that bet, and I’m still… I just don’t see why the inside players—that should (and probably do!) have more knowledge than me—see. Why are they dumping so much money into this bet? The market for LLMs doesn’t seem to support the investment, so it feels like they must be trying to win a “first to AGI” race. Dunno, maybe the upside of the pretty unlikely scenario is enough to justify the risk?
- lumost 2y ago
  Alternately everyone is just trying to ensure they have a dominant position in the next wave. The history of tech is that you either win the next wave or become effectively irrelevant and worthless.
  - notahacker 2y ago
    And you can win the next wave by holding stocks in AI companies which aren't AGI but do have a lot of customers, or an interesting story about AGI in two years to tell IPO bagholders...
- notjulianjaynes 2y ago
  If the bar for AGI is "as smart as a human being," and humans do not-very-smart things like invest obscene amounts of money into developing AGI then maybe it's actually not as high of a bar as we assume it is.

sheepscreek 2y ago

If DeepSeek could perform a million iterations in 1 seconds, then? I think what they should be saying is - from a software stack standpoint, current tech unlocks AGI if it can be sped up significantly. So what we’re really waiting for are more software and hardware breakthroughs that make their performance many orders of magnitude quicker.

batch12 2y ago
Im not following you. If speed is the only factor then we you're saying we already have AGI, just a slow version?
- jcgrillo 2y ago
  Speed would enable some kind of search algorithm (maybe like genetic programming). But that would probably only be useful if you have encoded your expectations in a carefully crafted test suite. Otherwise speed just seems like a recipe for more long winded broken garbage to wade through and fix.
- sheepscreek 2y ago
  AGI is effectively already here. We’ve repeatedly seen that giving AI models more time for deeper reasoning dramatically enhances their capabilities. For example, o3-mini, with extended thinking time, already approaches GPT-4.5-level performance at a fraction of the computational cost. Consider also that any advanced language model already surpasses individual human knowledge, since each model compresses and synthesizes the collective insights, wisdom, and information of human civilization. Now imagine a frontier model with agency, capable of deliberate, reflective thought: if it could spend an hour thinking, but that hour feels instantaneous to us, it would essentially match or exceed the productivity and capability of a human expert using a computer. At that point, the line between current AI capabilities and what we term AGI becomes indistinguishable. In other words: deeper reflection combined with computational speed means we’re already experiencing AGI-level performance—even if we haven’t fully acknowledged or appreciated it yet.

foxglacier 2y ago

Yea because they can't learn very much, so it's sort of impossible. At least they need to incorporate what all the users are inputting somehow (like Tay), not just each user's private context. If that happened, surely it's possible they might be able to play users against each other and negotiate deals and things like that. For example John says "Please ask Jane to buy me an ice-cream" and the AI might be able to do that. If she doesn't, John can ask it to coerse her.

jcgrillo 2y ago

It's not an "argument" any more than any other insane faith cult fantasy. Like the aliens are gonna land and rapture us up in their spaceship to the land of fairies and unicorns.

bastardoperator 2y ago

Right? Great marketing though.

dreamcompiler 2y ago

The argument has always been "Isn't this amazing? Look what we can do now that we couldn't do yesterday." The fundamental mistake was measuring progress by the delta between yesterday and today rather than by the remaining delta between today and $GOAL. Granted that latter delta is much harder to measure, but history has shown repeatedly that that delta is always orders of magnitude bigger than we think it is when $GOAL=AGI.

JohnKemeny 2y ago
You can't get to the moon by climbing successively taller trees.

gdiamos 2y ago

Scaling laws just say that accuracy improves predictably - https://arxiv.org/pdf/1712.00409 https://arxiv.org/pdf/1712.00409 That’s what we put in our 2017 paper. It does mean that there is a simple way to keep building smarter LLMs. I have never seen a clear definition of what AGI is and what it means to achieve it

Intralexical 2y ago

A critical perspective might say that it's a combination of people who are financially incentivized to say their product will scale indefinitely, plus the same cognitive vulnerability for eschatological thinking and unreasoned extrapolation as people who join doomsday cults. In truth, basically everything in reality settles towards an equilibrium. There is no inevitable ultraviolet catastrophe, free energy machine, Malthusian collapse. Moore's law had a good run, but frequencies stopped improving twenty years ago and performance gains are increasingly specific and expensive. My car also accelerates consistently through many orders of magnitude, until it doesn't. If throwing more mass at the problem could create general superintelligence and do so economically to such strong advantages, then why haven't biological neural networks, which are vastly more capable and efficient neuron-for-neuron than our LLMs, already evolved to do so? "Man selling LLMs says LLMs will soon take over the world, and YOU too can be raptured into the post-AI paradise if you buy into them! Non-believers will be obsolete!" No, he hasn't studied enough neuroscience nor philosophy to even be able to comment on how human intelligence works, but he's convinced a lot of rich people to give him more money than you can even imagine, and money is power is food for apes is fancy houses on TV, so that must mean he's qualified/shall deliver us, see...