4 ms·

> Given that predicting what happens next requires predicting what humans do, because humans are part of the world, if they keep getting better and better then

> Given that predicting what happens next requires predicting what humans do, because humans are part of the world, if they keep getting better and better then they'll be able to predict what humans would do well enough that they're capable of thinking any thoughts humans could think, because that's a requirement for predicting human behaviour. This doesn't follow. A just as likely (or even more likely) prediction is that they will fail to predict what happens next once it requires modeling humans. It won't get any more progress even if we increase training set size or model size - it will require an architecture change of some kind.

lucubratory 3y ago
Yes, it's completely possible that we get to some sort of limit and the scaling laws fall down, loss stops dropping even with more data, parameters, compute. But we have zero reason to believe that is the case, currently, it's only hypothetical. In terms of what we've experimentally seen so far, it just keeps getting better as you scale them up, so to find out if you're right and there's some future roadblock we don't know about we just have to keep scaling these things to see if it works. We haven't even seen loss functions start to level out - the closest argument I've seen to that is people misinterpreting the SGD curve (an artificial tapering we apply to the training process) as a fundamental architecture limitation rather than just an attempt to use resources wisely on a given training run. Basically, training of these models currently only costs a few million dollars, peanuts compared to the budgets of CERN or ITER, and we have zero evidence that we're approaching any sort of ceiling. If we keep scaling, maybe we see some heretofore unpredicted failure that breaks scaling laws like you're predicting. But maybe we don't, and the scaling laws (which have performed incredibly accurately so far) really do hold.
- kaba0 3y ago
  Having linear scaling up to a point is a far more reasonable guess, than a forever scaling function - as per Occam’s razor. If I have a rubbery string and try lengthening it, it will also show a linear scaling, until it catastrophically fails.
  - lucubratory 3y ago
    >Yes, it's completely possible that we get to some sort of limit and the scaling laws fall down, loss stops dropping even with more data, parameters, compute. But we have zero reason to believe that is the case, currently, it's only hypothetical [Edit:- and not really a good hypothetical, because there's no theoretical backing that explains current progress but predicts a future discontinuity]. In terms of what we've experimentally seen so far, it just keeps getting better as you scale them up, so to find out if you're right and there's some future roadblock we don't know about we just have to keep scaling these things to see if it works. Also I have no idea why you think Occam's Razor supports "There will be a discontinuity in this currently accurate observed trend, near enough in the future to be relevant" when the trend is talking about the effects of an algorithm applied to data. By analogy, it's something like claiming Moore's Law won't hold because Occam's Razor suggests at some point there's a physical breakdown, except instead of saying it now you're saying it in 1978, & you're saying it about an algorithmic output rather than a physical thing with an obvious limit (atomic size). Once we see any evidence of all of such a discontinuity, we can adjust what we're doing based on that. Until then, if we want to know whether scaling laws hold we should scale the models.
    - tsimionescu 3y ago
      First of all, there is no theory at all which explains the current progress of LLMs. It's not even a quantifiable progress - it's clearly there, but no one can say with any true precision whether GPT2 to GPT3 to GPT4 is following linear or exponential or whatever else precise function of progress in capabilities. We can clearly see that they are improving, but we have no idea how to accurately quantify this space. Secondly, we can see that there are obvious quantifiable physical increases in the requirements of training and inference for GPT4 compared to GPT3 and to GPT2. So, we can absolutely make a physical argument: at some point, you'll need too much energy to keep scaling up GPTs for it to be feasible, even if capability would keep improving with model size. Thirdly, the GPTs require ever more data to train, and there is a limited amount of text data in the world, of which they are already using a significant percent. So, there is also an argument that we might run out of data to improve even before we run out of physical resources. And making an argument that Moore's law will stop at some point due to physical constraints was just as correct in 1978 as it is in 2024. Predicting when that is is obviously hard - and the same is true for LLM scaling. We may be at the peek, or there may be 100 more generations before we even approach the limits I'm contemplating. But the limits are certainly there.

lucubratory 3y ago

Yes, it's completely possible that we get to some sort of limit and the scaling laws fall down, loss stops dropping even with more data, parameters, compute. But we have zero reason to believe that is the case, currently, it's only hypothetical. In terms of what we've experimentally seen so far, it just keeps getting better as you scale them up, so to find out if you're right and there's some future roadblock we don't know about we just have to keep scaling these things to see if it works. We haven't even seen loss functions start to level out - the closest argument I've seen to that is people misinterpreting the SGD curve (an artificial tapering we apply to the training process) as a fundamental architecture limitation rather than just an attempt to use resources wisely on a given training run. Basically, training of these models currently only costs a few million dollars, peanuts compared to the budgets of CERN or ITER, and we have zero evidence that we're approaching any sort of ceiling. If we keep scaling, maybe we see some heretofore unpredicted failure that breaks scaling laws like you're predicting. But maybe we don't, and the scaling laws (which have performed incredibly accurately so far) really do hold.

kaba0 3y ago
Having linear scaling up to a point is a far more reasonable guess, than a forever scaling function - as per Occam’s razor. If I have a rubbery string and try lengthening it, it will also show a linear scaling, until it catastrophically fails.
- lucubratory 3y ago
  >Yes, it's completely possible that we get to some sort of limit and the scaling laws fall down, loss stops dropping even with more data, parameters, compute. But we have zero reason to believe that is the case, currently, it's only hypothetical [Edit:- and not really a good hypothetical, because there's no theoretical backing that explains current progress but predicts a future discontinuity]. In terms of what we've experimentally seen so far, it just keeps getting better as you scale them up, so to find out if you're right and there's some future roadblock we don't know about we just have to keep scaling these things to see if it works. Also I have no idea why you think Occam's Razor supports "There will be a discontinuity in this currently accurate observed trend, near enough in the future to be relevant" when the trend is talking about the effects of an algorithm applied to data. By analogy, it's something like claiming Moore's Law won't hold because Occam's Razor suggests at some point there's a physical breakdown, except instead of saying it now you're saying it in 1978, & you're saying it about an algorithmic output rather than a physical thing with an obvious limit (atomic size). Once we see any evidence of all of such a discontinuity, we can adjust what we're doing based on that. Until then, if we want to know whether scaling laws hold we should scale the models.
  - tsimionescu 3y ago
    First of all, there is no theory at all which explains the current progress of LLMs. It's not even a quantifiable progress - it's clearly there, but no one can say with any true precision whether GPT2 to GPT3 to GPT4 is following linear or exponential or whatever else precise function of progress in capabilities. We can clearly see that they are improving, but we have no idea how to accurately quantify this space. Secondly, we can see that there are obvious quantifiable physical increases in the requirements of training and inference for GPT4 compared to GPT3 and to GPT2. So, we can absolutely make a physical argument: at some point, you'll need too much energy to keep scaling up GPTs for it to be feasible, even if capability would keep improving with model size. Thirdly, the GPTs require ever more data to train, and there is a limited amount of text data in the world, of which they are already using a significant percent. So, there is also an argument that we might run out of data to improve even before we run out of physical resources. And making an argument that Moore's law will stop at some point due to physical constraints was just as correct in 1978 as it is in 2024. Predicting when that is is obviously hard - and the same is true for LLM scaling. We may be at the peek, or there may be 100 more generations before we even approach the limits I'm contemplating. But the limits are certainly there.