4 ms·

When people say we can't get to AGI, they're typically either in the camp that there's something intrinsically special about us that is incapable of being repli

When people say we can't get to AGI, they're typically either in the camp that there's something intrinsically special about us that is incapable of being replicated, or various versions of the belief that it's too complex to be achieved within a timespan in which they would still recognize humanity. The former can be seen as a religious belief, but the latter is simply saying that we do not understand anywhere near enough about intelligence to develop AGI. You point to the unexpected leap in capability we've had recently, but fundamentally it isn't that different from what we have had for decades. The same fundamental unsolved limitations exist, such as of being unable to learn the way we do (eg if a child is writing a specific letter wrong, we correct that specific mistake, but that doesn't overwrite the knowledge of how to write the other letters). As a lecture I watched a while ago had put it, we've gotten pretty good at making the icing of the AGI cake recently, but we still have no idea what the cake is supposed to even look/taste like, so we barely know what ingredients we might need to make it let alone the actual steps involved.

logicchains 3y ago
>not understand anywhere near enough about intelligence to develop AGI. LLM training is essentially training them to predict what happens next. When we combine this with multi-modal training (video/audio), they're being trained to predict what happens next in the world. As we feed them more and more data and make them bigger and bigger, they'll get better and better at this task. Given that predicting what happens next requires predicting what humans do, because humans are part of the world, if they keep getting better and better then they'll be able to predict what humans would do well enough that they're capable of thinking any thoughts humans could think, because that's a requirement for predicting human behaviour. So just by training them on this one label, predict what happens next, we'd expect them to eventually develop human-level intelligence if the loss keeps decreasing.
- dotnet00 3y ago
  That doesn't really follow at all. It may result in something that is even better at faking the appearance of intelligence, but LLMs and similar stuff fundamentally lacks various features that make them not capable of becoming human level intelligences.
  - exe34 3y ago
    Humans are pretty good at faking intelligence too! It's only when you question them carefully and you yourself are an expert in the field that you can detect that they're confabulating. Conversely, I've never seen anybody produce anything but the next word coming out of their mouth. Therefore people are glorified stochastic parrots.
    - nradov 3y ago
      You're conflating knowledge of specific subject areas with generalized intelligence, including the ability to learn and metacognition.
      - exe34 3y ago
        What do you mean by learning and metacognition? An LLM can tell you whether or not something is in their context window. Humans regularly come up with "innovative" things that they actually saw from somewhere else and just happen to forget the source. Intelligence is like magic - every time a machine can do one thing on the list, it stops being intelligence, it's just engineering. Maybe intelligence really is just a big bag of tricks?
  - stevenhuang 3y ago
    > fundamentally lacks We don't actually know that yet. It may very well fundamentally have all that is necessary for intelligence but other pieces are missing. Frontier neuroscience and functional processes for how our brains work has more in common with the fundamentals of LLMs than not, vis-a-vis minimization of prediction errors.
    - kaba0 3y ago
      I don’t think “fundamentally having something” is actually meaningful here - as it’s enough to show it is Turing complete for it to have fundamentally everything. Nonetheless, it’s probably not excel that will become the next AGI.
- pshc 3y ago
  If the loss keeps decreasing forever, then that conclusion seems to follow. But it's likely there will be plateaus and ceilings that current neural net architectures cannot surpass. We likely need more breakthroughs to achieve enough generalized reasoning, reliable predictions, alignment, and scalability within hardware constraints.
- tsimionescu 3y ago
  > Given that predicting what happens next requires predicting what humans do, because humans are part of the world, if they keep getting better and better then they'll be able to predict what humans would do well enough that they're capable of thinking any thoughts humans could think, because that's a requirement for predicting human behaviour. This doesn't follow. A just as likely (or even more likely) prediction is that they will fail to predict what happens next once it requires modeling humans. It won't get any more progress even if we increase training set size or model size - it will require an architecture change of some kind.
  - lucubratory 3y ago
    Yes, it's completely possible that we get to some sort of limit and the scaling laws fall down, loss stops dropping even with more data, parameters, compute. But we have zero reason to believe that is the case, currently, it's only hypothetical. In terms of what we've experimentally seen so far, it just keeps getting better as you scale them up, so to find out if you're right and there's some future roadblock we don't know about we just have to keep scaling these things to see if it works. We haven't even seen loss functions start to level out - the closest argument I've seen to that is people misinterpreting the SGD curve (an artificial tapering we apply to the training process) as a fundamental architecture limitation rather than just an attempt to use resources wisely on a given training run. Basically, training of these models currently only costs a few million dollars, peanuts compared to the budgets of CERN or ITER, and we have zero evidence that we're approaching any sort of ceiling. If we keep scaling, maybe we see some heretofore unpredicted failure that breaks scaling laws like you're predicting. But maybe we don't, and the scaling laws (which have performed incredibly accurately so far) really do hold.
    - kaba0 3y ago
      Having linear scaling up to a point is a far more reasonable guess, than a forever scaling function - as per Occam’s razor. If I have a rubbery string and try lengthening it, it will also show a linear scaling, until it catastrophically fails.
marcyb5st 3y ago
My two cents on the topic. Currently, we are incapable of modeling causality effectively or at all (depends on your definition and to who you ask). LLMs seem to have it, but it's a mockery of causal reasoning. They use data created by us and so they can predict the next word following some of the inherent patterns we humans use which might look like they can reason to some extent. In this regard something like alphago/zero (in the way I understand them at least )are better than modern LLMs albeit in a very narrow field.
mannykannot 3y ago
The thing is that while the two camps exist among those who say we can't get to AGI, there is no real equivalent of the first camp among those who think it is possible. I'm not counting the people who say it has already (as good as) been achieved, as this claim is of the sort that should have been satisfactorily empirically verified before it is made.

logicchains 3y ago

>not understand anywhere near enough about intelligence to develop AGI. LLM training is essentially training them to predict what happens next. When we combine this with multi-modal training (video/audio), they're being trained to predict what happens next in the world. As we feed them more and more data and make them bigger and bigger, they'll get better and better at this task. Given that predicting what happens next requires predicting what humans do, because humans are part of the world, if they keep getting better and better then they'll be able to predict what humans would do well enough that they're capable of thinking any thoughts humans could think, because that's a requirement for predicting human behaviour. So just by training them on this one label, predict what happens next, we'd expect them to eventually develop human-level intelligence if the loss keeps decreasing.

dotnet00 3y ago
That doesn't really follow at all. It may result in something that is even better at faking the appearance of intelligence, but LLMs and similar stuff fundamentally lacks various features that make them not capable of becoming human level intelligences.
- exe34 3y ago
  Humans are pretty good at faking intelligence too! It's only when you question them carefully and you yourself are an expert in the field that you can detect that they're confabulating. Conversely, I've never seen anybody produce anything but the next word coming out of their mouth. Therefore people are glorified stochastic parrots.
  - nradov 3y ago
    You're conflating knowledge of specific subject areas with generalized intelligence, including the ability to learn and metacognition.
    - exe34 3y ago
      What do you mean by learning and metacognition? An LLM can tell you whether or not something is in their context window. Humans regularly come up with "innovative" things that they actually saw from somewhere else and just happen to forget the source. Intelligence is like magic - every time a machine can do one thing on the list, it stops being intelligence, it's just engineering. Maybe intelligence really is just a big bag of tricks?
- stevenhuang 3y ago
  > fundamentally lacks We don't actually know that yet. It may very well fundamentally have all that is necessary for intelligence but other pieces are missing. Frontier neuroscience and functional processes for how our brains work has more in common with the fundamentals of LLMs than not, vis-a-vis minimization of prediction errors.
  - kaba0 3y ago
    I don’t think “fundamentally having something” is actually meaningful here - as it’s enough to show it is Turing complete for it to have fundamentally everything. Nonetheless, it’s probably not excel that will become the next AGI.
pshc 3y ago
If the loss keeps decreasing forever, then that conclusion seems to follow. But it's likely there will be plateaus and ceilings that current neural net architectures cannot surpass. We likely need more breakthroughs to achieve enough generalized reasoning, reliable predictions, alignment, and scalability within hardware constraints.
tsimionescu 3y ago
> Given that predicting what happens next requires predicting what humans do, because humans are part of the world, if they keep getting better and better then they'll be able to predict what humans would do well enough that they're capable of thinking any thoughts humans could think, because that's a requirement for predicting human behaviour. This doesn't follow. A just as likely (or even more likely) prediction is that they will fail to predict what happens next once it requires modeling humans. It won't get any more progress even if we increase training set size or model size - it will require an architecture change of some kind.
- lucubratory 3y ago
  Yes, it's completely possible that we get to some sort of limit and the scaling laws fall down, loss stops dropping even with more data, parameters, compute. But we have zero reason to believe that is the case, currently, it's only hypothetical. In terms of what we've experimentally seen so far, it just keeps getting better as you scale them up, so to find out if you're right and there's some future roadblock we don't know about we just have to keep scaling these things to see if it works. We haven't even seen loss functions start to level out - the closest argument I've seen to that is people misinterpreting the SGD curve (an artificial tapering we apply to the training process) as a fundamental architecture limitation rather than just an attempt to use resources wisely on a given training run. Basically, training of these models currently only costs a few million dollars, peanuts compared to the budgets of CERN or ITER, and we have zero evidence that we're approaching any sort of ceiling. If we keep scaling, maybe we see some heretofore unpredicted failure that breaks scaling laws like you're predicting. But maybe we don't, and the scaling laws (which have performed incredibly accurately so far) really do hold.
  - kaba0 3y ago
    Having linear scaling up to a point is a far more reasonable guess, than a forever scaling function - as per Occam’s razor. If I have a rubbery string and try lengthening it, it will also show a linear scaling, until it catastrophically fails.

marcyb5st 3y ago

My two cents on the topic. Currently, we are incapable of modeling causality effectively or at all (depends on your definition and to who you ask). LLMs seem to have it, but it's a mockery of causal reasoning. They use data created by us and so they can predict the next word following some of the inherent patterns we humans use which might look like they can reason to some extent. In this regard something like alphago/zero (in the way I understand them at least )are better than modern LLMs albeit in a very narrow field.

mannykannot 3y ago

The thing is that while the two camps exist among those who say we can't get to AGI, there is no real equivalent of the first camp among those who think it is possible. I'm not counting the people who say it has already (as good as) been achieved, as this claim is of the sort that should have been satisfactorily empirically verified before it is made.