8 ms·

The Scaling Hypothesis (2021)

97 points by oli5679 4y ago

rossdavidh 4y ago
On the one hand, it is clearly true that much of the requirements for neural networks (in silicon) to achieve intelligence of a general sort, is to simply be big enough. On the other hand, anyone acquainted with the facts of biological brain size vs. intelligence, can see that this cannot be all that is going on. Bison are not clearly more intelligent than crows, not even when the part of the brain involved in operating the body parts and processing raw sensory data is excluded. Something other than scale must be involved, even if scale is a necessary condition.
- api 4y ago
  > not clearly more intelligent than crows There are two possibilities: they're not more intelligent than crows, or they are in a way that is not clear to us.
  - jeffreyrogers 4y ago
    Well brain size is clearly not the main driver behind intelligence because it's only roughly correlated with intelligence in humans.
    - api 4y ago
      There are some extreme cases too: https://www.rifters.com/crawl/?p=6116 https://www.rifters.com/crawl/?p=6116 That one is downright spooky, especially for its bizarre implications if you take it seriously.
      - FeepingCreature 4y ago
        Though see gwern's skeptical take. https://www.gwern.net/Hydrocephalus#problems-with-the-case-studies https://www.gwern.net/Hydrocephalus#problems-with-the-case-s...
- lumost 4y ago
  Fundamentally, it may not matter. Just like biological brains are intelligent in some way - an extremely large Language Model may be intelligent in some way. If current scaling trends hold, it may not be possible to distinguish a hypothetical GPT-6 vs. a human over reasonably sized conversations/text production tasks. Granted, such a GPT-6 model may be larger than any commercial application could support, and possibly beyond the range of research oriented funding as of today.
  - astrange 4y ago
    If GPT-6 was only larger, it would still only know the things in its training set, whereas a human can take arbitrary time to think, can go out and perform experiments, can Google/GPT things themselves, etc. GPT can never be an intelligent agent because it’s not an agent. Also, the increased training time to produce a really big model makes it less like an intelligence, which has low marginal cost to learn something new. The reason GPT looks intelligent is because you’re the one providing that; it’s just a big blob of inanimate knowledge.
    - lumost 4y ago
      While a LLM doesn't have the ability to perform real world experiments, it remains to be seen whether an LLM can "reason" over a contextual input, combined with its own outputs. There is no fundamental reason it can't, and no fundamental reason to think that it can. Anecdotal observation from AI Dungeon and similar tools would put this as a "maybe".
      - astrange 4y ago
        I believe it can do that, but there's two periods where it can "think". 1. During training time, which is very expensive in terms of input text and money - plus, model training often completely fails which is why you have to do checkpoints, hyper parameter searches, etc. 2. During inference, but it doesn't have arbitrary thinking and memory abilities like a human does, it only has however much space is in the input tokens + its weights. There are thoughts that can't fit even in an optimal model. GPT isn't just a model, it also has that sampling system which is a regular computer program (as opposed to a learned one), which does give it extra abilities.
- astrange 4y ago
  If bison were intelligent, they wouldn’t be cooperative considering how we genocided them - so it’d be hard to tell if they are or not.
  - pmoriarty 4y ago
    I think this is anthropomorphising them. We don't really know what other creatures think. The best we can do is make guesses based on their behavior. But it's still just a guess.
    - astrange 4y ago
      I think we have good evidence for animals learning not to cooperate with us, though; completely ignoring us is associated with animals living in uninhabited places, like penguins, and with ones we killed or ate to death, like historical emus and megafauna.
- dragonwriter 4y ago
  > On the other hand, anyone acquainted with the facts of biological brain size vs. intelligence, can see that this cannot be all that is going on. I don't think that's true, if big enough is measured in connections, not mass. > Bison are not clearly more intelligent than crows, not even when the part of the brain involved in operating the body parts and processing raw sensory data is excluded. While it's true that crows, despite a high EQ [0] for birds, are lower than some apparently less-intelligent large mammals, birds have lower cell size and thus potentially more connections than mass-based measures like EQ account for when comparing to mammals. So, it's not clear at all that network size (in surplus or ratio to basic functional support of the body) isn't the key thing for intelligence. [0] https://en.m.wikipedia.org/wiki/Encephalization_quotient https://en.m.wikipedia.org/wiki/Encephalization_quotient
light_hue_1 4y ago
As an ML researcher, simple back of the envelope math shows you that the scaling hypothesis is totally wrong. We will never reach human-level AI by simply scaling what we have today. GPT-3 has 10^11 parameters and needs 10^14 bytes of training data. Averaged performance on a bunch of benchmarks is 40-50% depending on what kind of prompts you provide: https://res.cloudinary.com/dyd911kmh/image/upload/f_auto,q_auto:best/v1598020447/gpt3-3_krvb14.png https://res.cloudinary.com/dyd911kmh/image/upload/f_auto,q_a... 10x fewer parameters drops your performance by about 10%. If you just linearly extrapolate that graph, and ML doesn't generally scale linearly, models tend to peter out eventually, you're talking about needing models that are 10^6 or more larger with a similar increase in training data. This is.. starting to be impractical. That's 10^17 or more parameters and 10^20 or more data. And that's assuming the models actually continue to learn. This is also extrapolating with an average. Datasets in machine learning are not difficulty calibrated at all. We have no idea how to measure difficulty. So this extrapolation is being driven by the easier datasets, and it won't saturate the hard ones. For example, GPT-3 makes a lot of systematic errors and there are plenty of benchmarks where it just isn't very good regardless of how many parameters it has. Our understanding of what intelligence is in the first place is the biggest hurdle here. This is why we can't benchmark systems. Why we can't come up with a benchmark, where performance on that benchmark means we're x% of the way toward an intelligent system. As systems get better, our benchmarks and datasets get better to keep up with them. So just saying we're going to saturate performance on today's benchmarks with some model that has 10^17 parameters just doesn't mean much at all. We have no guarantee and no reason to expect that doing well on today's benchmarks, even if we invested trillions of dollars would matter in the grand scheme of things. Doesn't mean these models can't be useful. But there's plenty more to do before we can just say "take what we have and invest $1T to scale it up and we'll be good to go".
- RandyRanderson 4y ago
  Also, we need to show that the data costs, dev costs, training costs, maint, etc, and inference time costs are cheaper than human alternatives for some reasonable loss function, for most tasks. If you look at the FN and FP rates for many tasks, the SoTA Transformer models are all VERY high vs humans and inference times are maybe only 10x. At 100x or 1000x the transistors, data, etc and even at 1/10 the loss, ML solutions are likely not competitive for many tasks.
Jack000 4y ago
I think the main disconnect between large language models and what people expect from "AI" is the fact that LMs learn meaningful representations whereas most people associate intelligence with meaningful behaviors. All examples of "natural" intelligence are embodied agents so it's difficult to imagine any other kind of intelligence. The large transformer models we have now are really the low hanging fruit. When you have a lot of data and compute, the easiest way to scale is to train a supervised model with ground truth labels. In contrast it's much harder to train an RL agent, where you have to design the environment and keep track of its state. Language modelling is a problem where you can get arbitrarily close to perfect but never actually achieve it. Without some kind of grounding in vision/proprioception there will always be gaps in understanding. When they start scaling GATO-like models I think we'll be a lot closer to human-like intelligence.
- sanxiyn 4y ago
  I agree this is what trips people up. Large language models do have a "goal" but it (predicting the next word) is very alien and intelligence is power to achieve goal so it can be only evaluated respective to goal. Let's say you are debating GPT-3. People evaluate debater's intelligence by how persuasive he is or how surprising things he say are (they are related, if you already know everything said you are unlikely to change opinion although there is such thing as different framing and perspective). This works because being persuasive is a reasonable guess at goal of a human debater. But entire goal of GPT-3 is in a sense not to be surprising! If you are surprising you will be worse at predicting the next word. Most training data is probably filled with neither side persuading anything, so being persuasive is also bad for predicting the next word. So this is why prompting works. If you want GPT-3 to simulate an intelligent human debater with goal of being persuasive, by default GPT-3's goal is misaligned with what you want. So try prefix like "debates are rarely decisive, but this debate turned out to feature surprising and incontrovertible argument by AI, let's see how it went" so GPT-3 can downweight abundant debates in training data where no such thing happens.
  - visarga 4y ago
    > entire goal of GPT-3 is in a sense not to be surprising! Well, no. You can raise the temperature to get surprising results, or lower it to get the default. GPT-3's job is to learn the distribution, you sample it how you prefer.
RodgerTheGreat 4y ago
Even if neural nets continue to come up with adequate solutions to challenging problems (sometimes, hopefully, maybe), at the end of the day the best-case scenario is a model that cannot be audited, holistically understood, or trusted with decisions of any gravity. If beating the strongest human Go players with a self-trained ML model was detonating a fission bomb, pervasive automation with zero accountability is a nuclear winter.
- canjobear 4y ago
  > at the end of the day the best-case scenario is a model that cannot be audited, holistically understood, or trusted with decisions of any gravity. So like a human being?
  - RodgerTheGreat 4y ago
    A human can be held responsible for a mistake. A model can't. For the ethically flexible, the latter is a benefit rather than a hazard: one can use ML to whitewash bias and rely upon public perception of computers as dispassionate and objective. Humans are expensive and require individual training. Models scale. In the time it would take a human to make a mistake, a broadly-deployed model might make millions of impactful mistakes.
    - sanxiyn 4y ago
      I think analogy with medicine is fruitful. Small molecules can't be held responsible either, but they can be recalled from market and they do. Our pharmacology isn't good enough to design, say, vaccine that works worse for Asians, but we did require clinical trial participants to be a representative sample. Medicine is also mass produced, that's why there is post-market surveillance to catch rare side effects missed in clinical trials. Even with all this analogy, we don't insist medicine should have known mechanism of action, although we do prefer it. So I think we will regulate models with recalls, testing standards, monitoring etc, but won't insist on understanding. By the way, don't we already have widely deployed models, such as PhotoDNA, which supposedly removes millions of images a year to filter child pornography? I wonder how it was evaluated to be suitable for deployment.
ZeroGravitas 4y ago
> The scaling hypothesis regards the blessings of scale as the secret of AGI: intelligence is ‘just’ simple neural units & learning algorithms applied to diverse experiences at a (currently) unreachable scale. As increasing computational resources permit running such algorithms at the necessary scale, the neural networks will get ever more intelligent. I think this is what I believe i.e. that animals and humans are just evolved machines, with no divine spark. Not sure that I agree that it's unpopular or that it allows you to make decades long predictions on progress that'll come true. I'm also not sure why you'd want to do this if smaller models are meeting your needs. Feels a bit like the future of flight being predicted as a man with flapping wings rather than a jet engine.
- abriosi 4y ago
  If you think humans are only interpolating machines then I agree with your statement. Personally, I don't think humans or animals are only capable of doing interpolation Nonetheless, assuming that these models are only doing interpolation it is astonishing Now, a question arises; is there some emerging weak extrapolation going on? I have no idea, until very recently I though I had a good grasp of what interpolation/extrapolation meant in "human conceptual space". Not anymore.
  - Valgrim 4y ago
    Can you explain what interpolation and extrapolation mean in this context?
    - abriosi 4y ago
      I wish I could do a good job at that. Let me rob a definition from this paper "Learning in High Dimension Always Amounts to Extrapolation" https://arxiv.org/pdf/2110.09485.pdf https://arxiv.org/pdf/2110.09485.pdf : Definition 1. Interpolation occurs for a sample x whenever this sample belongs to the convex hull of a set of samples X, {x1, . . . , xN }, if not, extrapolation occurs.
- paskozdilar 4y ago

97 points by oli5679 4y ago

On the one hand, it is clearly true that much of the requirements for neural networks (in silicon) to achieve intelligence of a general sort, is to simply be big enough. On the other hand, anyone acquainted with the facts of biological brain size vs. intelligence, can see that this cannot be all that is going on. Bison are not clearly more intelligent than crows, not even when the part of the brain involved in operating the body parts and processing raw sensory data is excluded. Something other than scale must be involved, even if scale is a necessary condition.

api 4y ago
> not clearly more intelligent than crows There are two possibilities: they're not more intelligent than crows, or they are in a way that is not clear to us.
- jeffreyrogers 4y ago
  Well brain size is clearly not the main driver behind intelligence because it's only roughly correlated with intelligence in humans.
  - api 4y ago
    There are some extreme cases too: https://www.rifters.com/crawl/?p=6116 https://www.rifters.com/crawl/?p=6116 That one is downright spooky, especially for its bizarre implications if you take it seriously.
    - FeepingCreature 4y ago
      Though see gwern's skeptical take. https://www.gwern.net/Hydrocephalus#problems-with-the-case-studies https://www.gwern.net/Hydrocephalus#problems-with-the-case-s...
lumost 4y ago
Fundamentally, it may not matter. Just like biological brains are intelligent in some way - an extremely large Language Model may be intelligent in some way. If current scaling trends hold, it may not be possible to distinguish a hypothetical GPT-6 vs. a human over reasonably sized conversations/text production tasks. Granted, such a GPT-6 model may be larger than any commercial application could support, and possibly beyond the range of research oriented funding as of today.
- astrange 4y ago
  If GPT-6 was only larger, it would still only know the things in its training set, whereas a human can take arbitrary time to think, can go out and perform experiments, can Google/GPT things themselves, etc. GPT can never be an intelligent agent because it’s not an agent. Also, the increased training time to produce a really big model makes it less like an intelligence, which has low marginal cost to learn something new. The reason GPT looks intelligent is because you’re the one providing that; it’s just a big blob of inanimate knowledge.
  - lumost 4y ago
    While a LLM doesn't have the ability to perform real world experiments, it remains to be seen whether an LLM can "reason" over a contextual input, combined with its own outputs. There is no fundamental reason it can't, and no fundamental reason to think that it can. Anecdotal observation from AI Dungeon and similar tools would put this as a "maybe".
    - astrange 4y ago
      I believe it can do that, but there's two periods where it can "think". 1. During training time, which is very expensive in terms of input text and money - plus, model training often completely fails which is why you have to do checkpoints, hyper parameter searches, etc. 2. During inference, but it doesn't have arbitrary thinking and memory abilities like a human does, it only has however much space is in the input tokens + its weights. There are thoughts that can't fit even in an optimal model. GPT isn't just a model, it also has that sampling system which is a regular computer program (as opposed to a learned one), which does give it extra abilities.
astrange 4y ago
If bison were intelligent, they wouldn’t be cooperative considering how we genocided them - so it’d be hard to tell if they are or not.
- pmoriarty 4y ago
  I think this is anthropomorphising them. We don't really know what other creatures think. The best we can do is make guesses based on their behavior. But it's still just a guess.
  - astrange 4y ago
    I think we have good evidence for animals learning not to cooperate with us, though; completely ignoring us is associated with animals living in uninhabited places, like penguins, and with ones we killed or ate to death, like historical emus and megafauna.
dragonwriter 4y ago
> On the other hand, anyone acquainted with the facts of biological brain size vs. intelligence, can see that this cannot be all that is going on. I don't think that's true, if big enough is measured in connections, not mass. > Bison are not clearly more intelligent than crows, not even when the part of the brain involved in operating the body parts and processing raw sensory data is excluded. While it's true that crows, despite a high EQ [0] for birds, are lower than some apparently less-intelligent large mammals, birds have lower cell size and thus potentially more connections than mass-based measures like EQ account for when comparing to mammals. So, it's not clear at all that network size (in surplus or ratio to basic functional support of the body) isn't the key thing for intelligence. [0] https://en.m.wikipedia.org/wiki/Encephalization_quotient https://en.m.wikipedia.org/wiki/Encephalization_quotient

light_hue_1 4y ago

As an ML researcher, simple back of the envelope math shows you that the scaling hypothesis is totally wrong. We will never reach human-level AI by simply scaling what we have today. GPT-3 has 10^11 parameters and needs 10^14 bytes of training data. Averaged performance on a bunch of benchmarks is 40-50% depending on what kind of prompts you provide: https://res.cloudinary.com/dyd911kmh/image/upload/f_auto,q_auto:best/v1598020447/gpt3-3_krvb14.png https://res.cloudinary.com/dyd911kmh/image/upload/f_auto,q_a... 10x fewer parameters drops your performance by about 10%. If you just linearly extrapolate that graph, and ML doesn't generally scale linearly, models tend to peter out eventually, you're talking about needing models that are 10^6 or more larger with a similar increase in training data. This is.. starting to be impractical. That's 10^17 or more parameters and 10^20 or more data. And that's assuming the models actually continue to learn. This is also extrapolating with an average. Datasets in machine learning are not difficulty calibrated at all. We have no idea how to measure difficulty. So this extrapolation is being driven by the easier datasets, and it won't saturate the hard ones. For example, GPT-3 makes a lot of systematic errors and there are plenty of benchmarks where it just isn't very good regardless of how many parameters it has. Our understanding of what intelligence is in the first place is the biggest hurdle here. This is why we can't benchmark systems. Why we can't come up with a benchmark, where performance on that benchmark means we're x% of the way toward an intelligent system. As systems get better, our benchmarks and datasets get better to keep up with them. So just saying we're going to saturate performance on today's benchmarks with some model that has 10^17 parameters just doesn't mean much at all. We have no guarantee and no reason to expect that doing well on today's benchmarks, even if we invested trillions of dollars would matter in the grand scheme of things. Doesn't mean these models can't be useful. But there's plenty more to do before we can just say "take what we have and invest $1T to scale it up and we'll be good to go".

RandyRanderson 4y ago
Also, we need to show that the data costs, dev costs, training costs, maint, etc, and inference time costs are cheaper than human alternatives for some reasonable loss function, for most tasks. If you look at the FN and FP rates for many tasks, the SoTA Transformer models are all VERY high vs humans and inference times are maybe only 10x. At 100x or 1000x the transistors, data, etc and even at 1/10 the loss, ML solutions are likely not competitive for many tasks.

Jack000 4y ago

I think the main disconnect between large language models and what people expect from "AI" is the fact that LMs learn meaningful representations whereas most people associate intelligence with meaningful behaviors. All examples of "natural" intelligence are embodied agents so it's difficult to imagine any other kind of intelligence. The large transformer models we have now are really the low hanging fruit. When you have a lot of data and compute, the easiest way to scale is to train a supervised model with ground truth labels. In contrast it's much harder to train an RL agent, where you have to design the environment and keep track of its state. Language modelling is a problem where you can get arbitrarily close to perfect but never actually achieve it. Without some kind of grounding in vision/proprioception there will always be gaps in understanding. When they start scaling GATO-like models I think we'll be a lot closer to human-like intelligence.

sanxiyn 4y ago
I agree this is what trips people up. Large language models do have a "goal" but it (predicting the next word) is very alien and intelligence is power to achieve goal so it can be only evaluated respective to goal. Let's say you are debating GPT-3. People evaluate debater's intelligence by how persuasive he is or how surprising things he say are (they are related, if you already know everything said you are unlikely to change opinion although there is such thing as different framing and perspective). This works because being persuasive is a reasonable guess at goal of a human debater. But entire goal of GPT-3 is in a sense not to be surprising! If you are surprising you will be worse at predicting the next word. Most training data is probably filled with neither side persuading anything, so being persuasive is also bad for predicting the next word. So this is why prompting works. If you want GPT-3 to simulate an intelligent human debater with goal of being persuasive, by default GPT-3's goal is misaligned with what you want. So try prefix like "debates are rarely decisive, but this debate turned out to feature surprising and incontrovertible argument by AI, let's see how it went" so GPT-3 can downweight abundant debates in training data where no such thing happens.
- visarga 4y ago
  > entire goal of GPT-3 is in a sense not to be surprising! Well, no. You can raise the temperature to get surprising results, or lower it to get the default. GPT-3's job is to learn the distribution, you sample it how you prefer.

RodgerTheGreat 4y ago

Even if neural nets continue to come up with adequate solutions to challenging problems (sometimes, hopefully, maybe), at the end of the day the best-case scenario is a model that cannot be audited, holistically understood, or trusted with decisions of any gravity. If beating the strongest human Go players with a self-trained ML model was detonating a fission bomb, pervasive automation with zero accountability is a nuclear winter.

canjobear 4y ago
> at the end of the day the best-case scenario is a model that cannot be audited, holistically understood, or trusted with decisions of any gravity. So like a human being?
- RodgerTheGreat 4y ago
  A human can be held responsible for a mistake. A model can't. For the ethically flexible, the latter is a benefit rather than a hazard: one can use ML to whitewash bias and rely upon public perception of computers as dispassionate and objective. Humans are expensive and require individual training. Models scale. In the time it would take a human to make a mistake, a broadly-deployed model might make millions of impactful mistakes.
  - sanxiyn 4y ago
    I think analogy with medicine is fruitful. Small molecules can't be held responsible either, but they can be recalled from market and they do. Our pharmacology isn't good enough to design, say, vaccine that works worse for Asians, but we did require clinical trial participants to be a representative sample. Medicine is also mass produced, that's why there is post-market surveillance to catch rare side effects missed in clinical trials. Even with all this analogy, we don't insist medicine should have known mechanism of action, although we do prefer it. So I think we will regulate models with recalls, testing standards, monitoring etc, but won't insist on understanding. By the way, don't we already have widely deployed models, such as PhotoDNA, which supposedly removes millions of images a year to filter child pornography? I wonder how it was evaluated to be suitable for deployment.

ZeroGravitas 4y ago

> The scaling hypothesis regards the blessings of scale as the secret of AGI: intelligence is ‘just’ simple neural units & learning algorithms applied to diverse experiences at a (currently) unreachable scale. As increasing computational resources permit running such algorithms at the necessary scale, the neural networks will get ever more intelligent. I think this is what I believe i.e. that animals and humans are just evolved machines, with no divine spark. Not sure that I agree that it's unpopular or that it allows you to make decades long predictions on progress that'll come true. I'm also not sure why you'd want to do this if smaller models are meeting your needs. Feels a bit like the future of flight being predicted as a man with flapping wings rather than a jet engine.

abriosi 4y ago
If you think humans are only interpolating machines then I agree with your statement. Personally, I don't think humans or animals are only capable of doing interpolation Nonetheless, assuming that these models are only doing interpolation it is astonishing Now, a question arises; is there some emerging weak extrapolation going on? I have no idea, until very recently I though I had a good grasp of what interpolation/extrapolation meant in "human conceptual space". Not anymore.
- Valgrim 4y ago
  Can you explain what interpolation and extrapolation mean in this context?
  - abriosi 4y ago
    I wish I could do a good job at that. Let me rob a definition from this paper "Learning in High Dimension Always Amounts to Extrapolation" https://arxiv.org/pdf/2110.09485.pdf https://arxiv.org/pdf/2110.09485.pdf : Definition 1. Interpolation occurs for a sample x whenever this sample belongs to the convex hull of a set of samples X, {x1, . . . , xN }, if not, extrapolation occurs.
paskozdilar 4y ago