7 ms·

>All evidence suggests that current LLMs don't scale (EDIT: they do scale, but at a certain point somewhere around GPT4 the scaling slows down very quickly) Th

>All evidence suggests that current LLMs don't scale (EDIT: they do scale, but at a certain point somewhere around GPT4 the scaling slows down very quickly) There seems to actually be no evidence that LLMs dont scale. All data from LLMs leading up to GPT4 indicates they scale incredibly well actually. Do you have a single paper you can point to that says otherwise?

4death4 3y ago
No OP, but no amount of data will ever enable an LLM to know when it doesn’t know something, for instance. That’s a fundamental limitation with the current architecture.
- bilbo0s 3y ago
  But that's different from scaling. You can not know what you don't know, and learn gobs of new stuff at the same time. That's the natural state of most learners.
  - deleted 3y ago
    [deleted]
  - datameta 3y ago
    When we have an external source of information that is not self-accountable, it is different from when we are both the learner and functor - when we notice mistakes we learn from them. We're always looking out for errors, and if they happen then (usually) we alone feel responsible for their effects.
    - bilbo0s 3y ago
      Again, you're talking about something different than scaling. There are many different aspects of emotions, life, or even learning. But the comment was about scaling learning, which can be unaffected by other aspects, such as whether you learned something incorrect. More succinctly: I can still teach Simpson's Rule in a Calculus class to a young man or woman who subscribes to Creationism, or the Flat Earth theory. None of their false knowledge affects their innate ability to take in new knowledge.
      - 4death4 3y ago
        But what is the purpose of "scaling?" Broadly speaking, it means that there is a marked and obtainable improvement on an objective function in response to additional data. But for certain criteria, like the possession of knowledge, it appears attention-based LLMs aren't capable of that, regardless the amount of data. So they don't "scale" if you use that as an objective function.
- IanCal 3y ago
  They can say they don't know things already. How are you defining "know" here?
  - nyrikki 3y ago
    Ah the fun of known knowns, known unknowns, unknown unknowns, and weak negation. https://blog.research.google/2024/01/can-large-language-models-identify-and.html https://blog.research.google/2024/01/can-large-language-mode... Negation in the training data, exhaustion, or unknowable unknowns like future events is about an LLM will ever be able to say it "doesn't know" reliably. LLMs have no concept of truth or correctness really, they are just probabilistic appending to output to match learned ideal patterns.
    - humansareok1 3y ago
      >LLMs have no concept of truth or correctness really, they are just probabilistic appending to output to match learned ideal patterns. I think they do have those concepts, like they have many others but they are not bound to them in any way.
  - fhd2 3y ago
    They're trained (and possibly scripted) to not "know" things or not be able to do things. But at their core, they're going to generate the most plausible continuation, even if that isn't very plausible (like non-existent functions). I think this can be improved (playing with the temperature in self hosted models depending on the nature of the task e.g.), but it doesn't seem fully solvable with current LLMs.
geysersam 3y ago
If they need 100x the available training data to scale beyond their current capabilities they clearly don't scale particularly well.
- famouswaffles 3y ago
  Good thing they don't then
  - geysersam 3y ago
    I don't understand what you mean. To improve performance using the same basic architecture the model need to scale both compute and training data. Where are we going to find another 10x the web sized text training corpus?
    - famouswaffles 3y ago
      Gpt-4 was allegedly trained on 6T tokens for 2 epochs (i.e twice). 6T is far from exhausting the text data we have. Here's a 5 language 30T token dataset from web scrapped data. https://github.com/togethercomputer/RedPajama-Data https://github.com/togethercomputer/RedPajama-Data And that is just web scrapped data. There are trillions of valuable tokens worth of text from the likes of pdfs/ebooks, academic journals and other documents that essentially has no web presence otherwise. https://annas-archive.org/llm https://annas-archive.org/llm >To improve performance using the same basic architecture the model need to scale both compute and training data. What you are trying to say here is that you need to scale both parameters and data as scaling data increases compute. That said, it's not really true. It would be best if you scaled both data and parameters but you can get increased performance just scaling one or the other.
      - geysersam 3y ago
        The 6T token dataset is surely a high quality subset / refined extract from much larger public datasets. It's misleading to compare their sizes directly.
darkoob12 3y ago
I think the main argument is based on diminishing returns. They expected larger improvement in performance given the 5x increase in number of parameters.
nicklecompte 3y ago
There was an article on HN last week discussing theoretical proof that scaling has a hard limit: https://www.quantamagazine.org/how-chain-of-thought-reasoning-helps-neural-networks-compute-20240321/ https://www.quantamagazine.org/how-chain-of-thought-reasonin... Specifically: > To study how the parallel structure of transformers might limit their capabilities, the pair considered the case where transformers didn’t feed their output back into their input — instead, their first output would have to be the final answer. They proved that the transformers in this theoretical framework couldn’t solve any computational problems that lie outside a specific complexity class. And many math problems, including relatively simple ones like solving linear equations, are thought to lie outside this class. > But no matter how a prompt is phrased, as long as it causes a language model to output step-by-step solutions, the model can in principle reuse the results of intermediate steps on subsequent passes through the transformer. That could provide a way to evade the limits of parallel computation... They quantified how that extra computational power depends on the number of intermediate steps a transformer is allowed to use before it must spit out a final answer. In general, researchers expect the appropriate number of intermediate steps for solving any problem to depend on the size of the input to the problem. ...Indeed, Merrill and Sabharwal proved that chain of thought only really begins to help when the number of intermediate steps grows in proportion to the size of the input, and many problems require the number of intermediate steps to grow much larger still. So there’s a large space of O(n) and O(constant) problems that LLMs seem to scale very well on, but if they need O(n^2) examples in their training data to learn how to solve O(n^2) problems (even with prompt engineering!) then that’s a hard limit. ETA: there’s also good (IMO irrefutable) evidence that AI researchers have deluded themselves about the scalability of LLMs because of motivated statistical reasoning: https://arxiv.org/abs/2304.15004 https://arxiv.org/abs/2304.15004 In fact it seems to me that “LLMs scale well” means “LLMs can solve O(n) problems if you give them O(n) examples in their training data,” which is actually not very interesting. The interesting thing is that the internet + Moore’s law allows us to build (and compute on) O(n) training data for an enormous variety of problems.
- humansareok1 3y ago
  >> To study how the parallel structure of transformers might limit their capabilities, the pair considered the case where transformers didn’t feed their output back into their input — instead, their first output would have to be the final answer. They proved that the transformers in this theoretical framework couldn’t solve any computational problems that lie outside a specific complexity class. And many math problems, including relatively simple ones like solving linear equations, are thought to lie outside this class. This seems like somewhat of an artificial constraint? E.g. we can explicitly ask GPT4 to solve problems using code which I wouldn't think has the same limits. The model might not be able to solve a problem directly but can in code. >ETA: there’s also good (IMO irrefutable) evidence that AI researchers have deluded themselves about the scalability of LLMs because of motivated statistical reasoning: https://arxiv.org/abs/2304.15004 https://arxiv.org/abs/2304.15004 Curious to hear how this relates to scaling. I think a lot of the emergence dialog is just semantics. I.e. if GPT3 can only score a 50% on a 3 digit arithmetic test but GPT4 scores 80% this 'ability' might not literally be emergent but functionally it is, at a larger scale it 'gained' the ability to do 3 digit arithmetic. It's not clear how this indicates issues with Scalability though. For example in the paper the authors state: "However, if one changes from nonlinear Accuracy to linear Token Edit Distance while keeping the models’ outputs fixed, the family’s performance smoothly, continuously and predictably improves with increasing scale"

4death4 3y ago

No OP, but no amount of data will ever enable an LLM to know when it doesn’t know something, for instance. That’s a fundamental limitation with the current architecture.

bilbo0s 3y ago
But that's different from scaling. You can not know what you don't know, and learn gobs of new stuff at the same time. That's the natural state of most learners.
- deleted 3y ago
  [deleted]
- datameta 3y ago
  When we have an external source of information that is not self-accountable, it is different from when we are both the learner and functor - when we notice mistakes we learn from them. We're always looking out for errors, and if they happen then (usually) we alone feel responsible for their effects.
  - bilbo0s 3y ago
    Again, you're talking about something different than scaling. There are many different aspects of emotions, life, or even learning. But the comment was about scaling learning, which can be unaffected by other aspects, such as whether you learned something incorrect. More succinctly: I can still teach Simpson's Rule in a Calculus class to a young man or woman who subscribes to Creationism, or the Flat Earth theory. None of their false knowledge affects their innate ability to take in new knowledge.
    - 4death4 3y ago
      But what is the purpose of "scaling?" Broadly speaking, it means that there is a marked and obtainable improvement on an objective function in response to additional data. But for certain criteria, like the possession of knowledge, it appears attention-based LLMs aren't capable of that, regardless the amount of data. So they don't "scale" if you use that as an objective function.
IanCal 3y ago
They can say they don't know things already. How are you defining "know" here?
- nyrikki 3y ago
  Ah the fun of known knowns, known unknowns, unknown unknowns, and weak negation. https://blog.research.google/2024/01/can-large-language-models-identify-and.html https://blog.research.google/2024/01/can-large-language-mode... Negation in the training data, exhaustion, or unknowable unknowns like future events is about an LLM will ever be able to say it "doesn't know" reliably. LLMs have no concept of truth or correctness really, they are just probabilistic appending to output to match learned ideal patterns.
  - humansareok1 3y ago
    >LLMs have no concept of truth or correctness really, they are just probabilistic appending to output to match learned ideal patterns. I think they do have those concepts, like they have many others but they are not bound to them in any way.
- fhd2 3y ago
  They're trained (and possibly scripted) to not "know" things or not be able to do things. But at their core, they're going to generate the most plausible continuation, even if that isn't very plausible (like non-existent functions). I think this can be improved (playing with the temperature in self hosted models depending on the nature of the task e.g.), but it doesn't seem fully solvable with current LLMs.

geysersam 3y ago

If they need 100x the available training data to scale beyond their current capabilities they clearly don't scale particularly well.

famouswaffles 3y ago
Good thing they don't then
- geysersam 3y ago
  I don't understand what you mean. To improve performance using the same basic architecture the model need to scale both compute and training data. Where are we going to find another 10x the web sized text training corpus?
  - famouswaffles 3y ago
    Gpt-4 was allegedly trained on 6T tokens for 2 epochs (i.e twice). 6T is far from exhausting the text data we have. Here's a 5 language 30T token dataset from web scrapped data. https://github.com/togethercomputer/RedPajama-Data https://github.com/togethercomputer/RedPajama-Data And that is just web scrapped data. There are trillions of valuable tokens worth of text from the likes of pdfs/ebooks, academic journals and other documents that essentially has no web presence otherwise. https://annas-archive.org/llm https://annas-archive.org/llm >To improve performance using the same basic architecture the model need to scale both compute and training data. What you are trying to say here is that you need to scale both parameters and data as scaling data increases compute. That said, it's not really true. It would be best if you scaled both data and parameters but you can get increased performance just scaling one or the other.
    - geysersam 3y ago
      The 6T token dataset is surely a high quality subset / refined extract from much larger public datasets. It's misleading to compare their sizes directly.

darkoob12 3y ago

I think the main argument is based on diminishing returns. They expected larger improvement in performance given the 5x increase in number of parameters.

nicklecompte 3y ago

There was an article on HN last week discussing theoretical proof that scaling has a hard limit: https://www.quantamagazine.org/how-chain-of-thought-reasoning-helps-neural-networks-compute-20240321/ https://www.quantamagazine.org/how-chain-of-thought-reasonin... Specifically: > To study how the parallel structure of transformers might limit their capabilities, the pair considered the case where transformers didn’t feed their output back into their input — instead, their first output would have to be the final answer. They proved that the transformers in this theoretical framework couldn’t solve any computational problems that lie outside a specific complexity class. And many math problems, including relatively simple ones like solving linear equations, are thought to lie outside this class. > But no matter how a prompt is phrased, as long as it causes a language model to output step-by-step solutions, the model can in principle reuse the results of intermediate steps on subsequent passes through the transformer. That could provide a way to evade the limits of parallel computation... They quantified how that extra computational power depends on the number of intermediate steps a transformer is allowed to use before it must spit out a final answer. In general, researchers expect the appropriate number of intermediate steps for solving any problem to depend on the size of the input to the problem. ...Indeed, Merrill and Sabharwal proved that chain of thought only really begins to help when the number of intermediate steps grows in proportion to the size of the input, and many problems require the number of intermediate steps to grow much larger still. So there’s a large space of O(n) and O(constant) problems that LLMs seem to scale very well on, but if they need O(n^2) examples in their training data to learn how to solve O(n^2) problems (even with prompt engineering!) then that’s a hard limit. ETA: there’s also good (IMO irrefutable) evidence that AI researchers have deluded themselves about the scalability of LLMs because of motivated statistical reasoning: https://arxiv.org/abs/2304.15004 https://arxiv.org/abs/2304.15004 In fact it seems to me that “LLMs scale well” means “LLMs can solve O(n) problems if you give them O(n) examples in their training data,” which is actually not very interesting. The interesting thing is that the internet + Moore’s law allows us to build (and compute on) O(n) training data for an enormous variety of problems.

humansareok1 3y ago
>> To study how the parallel structure of transformers might limit their capabilities, the pair considered the case where transformers didn’t feed their output back into their input — instead, their first output would have to be the final answer. They proved that the transformers in this theoretical framework couldn’t solve any computational problems that lie outside a specific complexity class. And many math problems, including relatively simple ones like solving linear equations, are thought to lie outside this class. This seems like somewhat of an artificial constraint? E.g. we can explicitly ask GPT4 to solve problems using code which I wouldn't think has the same limits. The model might not be able to solve a problem directly but can in code. >ETA: there’s also good (IMO irrefutable) evidence that AI researchers have deluded themselves about the scalability of LLMs because of motivated statistical reasoning: https://arxiv.org/abs/2304.15004 https://arxiv.org/abs/2304.15004 Curious to hear how this relates to scaling. I think a lot of the emergence dialog is just semantics. I.e. if GPT3 can only score a 50% on a 3 digit arithmetic test but GPT4 scores 80% this 'ability' might not literally be emergent but functionally it is, at a larger scale it 'gained' the ability to do 3 digit arithmetic. It's not clear how this indicates issues with Scalability though. For example in the paper the authors state: "However, if one changes from nonlinear Accuracy to linear Token Edit Distance while keeping the models’ outputs fixed, the family’s performance smoothly, continuously and predictably improves with increasing scale"