7 ms·

A quick summary/translation for those of us who don't speak ML. We keep hearing about these giant models like GPT3 with 1.5 billion paramaters. Parameters are

A quick summary/translation for those of us who don't speak ML. We keep hearing about these giant models like GPT3 with 1.5 billion paramaters. Parameters are the things that change when we train a model, you can think about them as degrees of freedom. If you have a lot of parameters, theory made us believe that the model would just "overfit" the training data, e.g. memorize it. That's bad, because when new data comes in in production we'd expect the model to not be able to "generalize" to it, e.g. make accurate predictions on data it hasn't seen before, because it's just memorized training data instead of uncovering the "guiding principles" of the data so to speak. In practice, these huge models are, in laymans terms, fucking awesome and work really well e.g. they generalize and work in production. No one understands why. This paper is a survey or overview of what "too many paramaters" are, and all the research into why these models work even though they shouldn't.

aomobile 5y ago
Thanks for the summary!
Salgat 5y ago
To add to this, there's a misleading phenomenon that first occurs where the performance actually gets worse with too much data/parameters/epochs, but oddly improves again if you throw even more at the model.
- nexuist 5y ago
  Is this the ML equivalent of Dunning–Kruger effect? A model with a bit of data is too afraid of being wrong to be overconfident. A model with a bit more data is overconfident in itself and gets things wrong. Finally, a model with tons and tons of data understands the complexity of the problem set and once again becomes too afraid of being wrong.
  - visarga 5y ago
    Model confidence as reported by softmax probability scores is notoriously noisy and miscalibrated. With larger models and more data the confidence estimation gets more nuanced.
- jointpdf 5y ago
  For the interested, this phenomenon is known as (deep) double descent: https://openai.com/blog/deep-double-descent/ https://openai.com/blog/deep-double-descent/ https://www.lesswrong.com/posts/FRv7ryoqtvSuqBxuT/understanding-deep-double-descent https://www.lesswrong.com/posts/FRv7ryoqtvSuqBxuT/understand... (Edit: Oh, the definition appears in the abstract of the linked paper.)
sdenton4 5y ago
My big beef with a lot of the 'leading edge' ML research is that it tends to be waaaaay too focused on classification problems, and ImageNet in particular. And, last I checked, you /do/ still fight with overfitting in classification models, by cleverly choosing learning rate schedules and using early stopping schemes, 'double descent' be damned. You can solve classification with a hash function: Hash the image, and then just memorize which label goes with which hash. You can try to dodge this obviously dodgy solution by adding augmentation to the dataset. Then you instead learn to find a representation invariant under the set of augmentations, and learn the hash of that representation. It turns out these augmentation-invariant representations are actually pretty good, so we can solve the classification problem in what looks like a general way. However, there are many other classes of problems where the hash problem doesn't exist, because the information density of the outputs is too high to memorize in the same way. Specifically, generative models, and the sorts of predictive/infill problems used for self-supervision. In these spaces, the problems are more like: "Given this pile of augmented input, generate half a megabyte of coherent output." These kinds of problems simply don't overfit: Train a speech separation model on a big dataset, and the train+eval quality metrics will just asymptote their way up and to the right until you run out of training budget.
- deleted 5y ago
  [deleted]
- quocanh 5y ago
  When you say hash function, do you mean a cryptographic hash function? How on earth could the performance of that be anywhere near the simplest probabilistic algorithm on unseen examples?
  - sdenton4 5y ago
    No, nothing cryptographic here. All I'm saying is that you can memorize the dataset by extracting a small fingerprint of each training example and associating it with an output label: ie, learn by lookup table. Then you don't need to memorize the whole training set, you just need to find/learn the fingerprinting function. With no augmentation, you might as well use MD5... With augmentation, you do need to do some actual work to learn to extract an augmentation invariant projection of the training examples, but the basic principle is the same.
diffCtx 5y ago
My take as a 90s math grad (out of touch with modern teaching): Theory is useful to show human society is stagnant. There’s an infinite number of sentences but our ML models are having tons of “success” as society relies on finite set in daily life; those that instigate commerce. Like religion relied on an acceptable finite set of sentences, so too does our society. We’re a bunch of weird little missionaries living in one geometric world, still believing in bigger purpose. ML isn’t really outputting novelty, it’s spewing our own inanity at us, and helping correct some bad math in engineering cases. We’re easily mesmerized apes.
baron_harkonnen 5y ago
> In practice, these huge models are, in laymans terms, fucking awesome and work really well A similarly surprising result from an adjacent community, Bayesian Statistics, is that in the case of hierarchical models, increasing your number of parameters can paradoxically reduce overfitting. The scale of parameters in Bayesian model's is no where near that of these deep neural nets, but nonetheless this is a similarly shocking result since typically adding parameters is penalized when model building. It's a bit more explainable in Bayesian stats since what you're using some parameters for is limiting the impact of more granular parameters (i.e. you're learning a prior probability distribution for the other parameters, which prevents extreme overfitting in cases with less information). I wouldn't be too surprised if eventually we realized there was a similar cause preventing overfitting is overparameterized ml models.
- naomisperfume 5y ago
  Do you have any good references of this phenomenon in hierarchical models?
  - deleted 5y ago
    [deleted]
  - bigfudge 5y ago
    German and Hill has a brief intro and some references.
- talolard 5y ago
  I don’t know if it’s correct , but I often think of a classification model as learning the parameters of a dirchlet distribution with the final softmax layer being a sample from it
- sjg007 5y ago
  This is likely part of the reason. The only problem is said models require a lot of data but Humans can learn from a very small number of examples.
yldedly 5y ago
>In practice, these huge models are, in laymans terms, fucking awesome and work really well e.g. they generalize and work in production. No one understands why. To add nuance to this, these models are awesome at interpolation, but not so much at extrapolation. Or in different terms, they generalize very well to an IID test set, but don't generalize under (even slight) distribution shift. The main reason for this is that these models tend to solve classification and regression problem quite differently from how humans do it. Broadly speaking, a large, flexible NN will find a "shortcut", i.e. a simple relation between some part of the input and the output, which may not be informative in the way we want; such as a watermark in the corner of an image, or statistical regularities in textures which disappear in slightly different lighting conditions. See e.g. https://thegradient.pub/shortcuts-neural-networks-love-to-cheat/ https://thegradient.pub/shortcuts-neural-networks-love-to-ch... I think it's fair to say that these models are great when you have an enormous dataset that covers the entire domain, but sub-Google-scale problems are usually still solved by underparametrized models (even at Google).
- rich_sasha 5y ago
  It depends. It really doesn’t take that much data to train a pretty stunning (if simple) RNN character-level “language model” that beats any n-gram. Or on mnist. ANNs really are a useful tool for a vast class of problems, many of which can be solved with comparatively little data. Maybe your point stands, and it’s just that some domains need less data, just saying.
  - yldedly 5y ago
    >ANNs really are a useful tool for a vast class of problems, many of which can be solved with comparatively little data. For sure, it all depends on how robust the model needs to be, how strongly it needs to generalize. If your dataset covers the entire domain, you don't need a robust model. If you need strong generalization, then you need to build in stronger priors. Take f(x) = x^2. If your model only needs to work in finite interval, you just need a decent sample that covers that interval. But if it needs to generalize outside that interval, no amount of parameters will give you good performance. Outside the boundaries of the interval, the NN will either be constant (with a sigmoid activation) or linear (with ReLU type activations).
andrewnc 5y ago
Super pedantic comment. GPT-3 has 175 Billion parameters. GPT-2 was the 1.5 Billion model.
988747 5y ago
> In practice, these huge models are, in laymans terms, fucking awesome and work really well e.g. they generalize and work in production. No one understands why. How about the resulting weights? If most of them are close to 0, then that would mean that a part of the training is for NN to learn which of 1.5B parameters are relevant, and which are not.
- rich_sasha 5y ago
  There is something called the golden ticket theory (maybe mentioned in the paper, I’m on my phone), that says indeed that the large models are effectively ensembles of massive random models, and the top levels of the network pick the one or two that randomly happen to work. Maybe true but even then only part of the story, kernels in CNN genuinely seem to learn features like edges and textures.
- talolard 5y ago
  There are two answers to this. First, empirically we see that the more parameters we add the better the model performs ==> Weights continue to contribute (and aren't dead) . Second, there is a very popular paper called "The lottery ticket hypothesis" [1] that in any network you can find subnetworks that work just as well. e.g. The parameters are redundant. This was written in 2018, which is a long time ago in big NN world, so I'm not sure how it holds up to current insanity sized models. [1]https://arxiv.org/abs/1803.03635 https://arxiv.org/abs/1803.03635
  - sdenton4 5y ago
    A couple notes... 1) Imagine the loss surface of a given model architecture; each point on the surface corresponds to a full set of weights, and the value at the point is the model loss. So, a billion-dimensional surface, give or take. There's a massive amount of flexibility in that space. Some models in the surface are sparse, but they are adjacent to models which are just as good but not sparse at all. Likewise, if you 'rotate' a sparse model, you can end up with an entirely equivalent dense model. So, you really need additional 'pressure' on the learning problem to ensure you actually get sparsity, even if the sparsity is in some sense natural. 2) IIUC, lottery ticket kinda breaks with larger models/problems. For small enough problems, the initial random projection given by the random starting weights is already good enough to build on. For bigger + more complicated problems, you need to really adapt in early training, and so lottery ticket breaks down.
deleted 5y ago
[deleted]

aomobile 5y ago

Thanks for the summary!

Salgat 5y ago

To add to this, there's a misleading phenomenon that first occurs where the performance actually gets worse with too much data/parameters/epochs, but oddly improves again if you throw even more at the model.

nexuist 5y ago
Is this the ML equivalent of Dunning–Kruger effect? A model with a bit of data is too afraid of being wrong to be overconfident. A model with a bit more data is overconfident in itself and gets things wrong. Finally, a model with tons and tons of data understands the complexity of the problem set and once again becomes too afraid of being wrong.
- visarga 5y ago
  Model confidence as reported by softmax probability scores is notoriously noisy and miscalibrated. With larger models and more data the confidence estimation gets more nuanced.
jointpdf 5y ago
For the interested, this phenomenon is known as (deep) double descent: https://openai.com/blog/deep-double-descent/ https://openai.com/blog/deep-double-descent/ https://www.lesswrong.com/posts/FRv7ryoqtvSuqBxuT/understanding-deep-double-descent https://www.lesswrong.com/posts/FRv7ryoqtvSuqBxuT/understand... (Edit: Oh, the definition appears in the abstract of the linked paper.)

sdenton4 5y ago

My big beef with a lot of the 'leading edge' ML research is that it tends to be waaaaay too focused on classification problems, and ImageNet in particular. And, last I checked, you /do/ still fight with overfitting in classification models, by cleverly choosing learning rate schedules and using early stopping schemes, 'double descent' be damned. You can solve classification with a hash function: Hash the image, and then just memorize which label goes with which hash. You can try to dodge this obviously dodgy solution by adding augmentation to the dataset. Then you instead learn to find a representation invariant under the set of augmentations, and learn the hash of that representation. It turns out these augmentation-invariant representations are actually pretty good, so we can solve the classification problem in what looks like a general way. However, there are many other classes of problems where the hash problem doesn't exist, because the information density of the outputs is too high to memorize in the same way. Specifically, generative models, and the sorts of predictive/infill problems used for self-supervision. In these spaces, the problems are more like: "Given this pile of augmented input, generate half a megabyte of coherent output." These kinds of problems simply don't overfit: Train a speech separation model on a big dataset, and the train+eval quality metrics will just asymptote their way up and to the right until you run out of training budget.

deleted 5y ago
[deleted]
quocanh 5y ago
When you say hash function, do you mean a cryptographic hash function? How on earth could the performance of that be anywhere near the simplest probabilistic algorithm on unseen examples?
- sdenton4 5y ago
  No, nothing cryptographic here. All I'm saying is that you can memorize the dataset by extracting a small fingerprint of each training example and associating it with an output label: ie, learn by lookup table. Then you don't need to memorize the whole training set, you just need to find/learn the fingerprinting function. With no augmentation, you might as well use MD5... With augmentation, you do need to do some actual work to learn to extract an augmentation invariant projection of the training examples, but the basic principle is the same.

diffCtx 5y ago

My take as a 90s math grad (out of touch with modern teaching): Theory is useful to show human society is stagnant. There’s an infinite number of sentences but our ML models are having tons of “success” as society relies on finite set in daily life; those that instigate commerce. Like religion relied on an acceptable finite set of sentences, so too does our society. We’re a bunch of weird little missionaries living in one geometric world, still believing in bigger purpose. ML isn’t really outputting novelty, it’s spewing our own inanity at us, and helping correct some bad math in engineering cases. We’re easily mesmerized apes.

baron_harkonnen 5y ago

> In practice, these huge models are, in laymans terms, fucking awesome and work really well A similarly surprising result from an adjacent community, Bayesian Statistics, is that in the case of hierarchical models, increasing your number of parameters can paradoxically reduce overfitting. The scale of parameters in Bayesian model's is no where near that of these deep neural nets, but nonetheless this is a similarly shocking result since typically adding parameters is penalized when model building. It's a bit more explainable in Bayesian stats since what you're using some parameters for is limiting the impact of more granular parameters (i.e. you're learning a prior probability distribution for the other parameters, which prevents extreme overfitting in cases with less information). I wouldn't be too surprised if eventually we realized there was a similar cause preventing overfitting is overparameterized ml models.

naomisperfume 5y ago
Do you have any good references of this phenomenon in hierarchical models?
- deleted 5y ago
  [deleted]
- bigfudge 5y ago
  German and Hill has a brief intro and some references.
talolard 5y ago
I don’t know if it’s correct , but I often think of a classification model as learning the parameters of a dirchlet distribution with the final softmax layer being a sample from it
sjg007 5y ago
This is likely part of the reason. The only problem is said models require a lot of data but Humans can learn from a very small number of examples.

yldedly 5y ago

>In practice, these huge models are, in laymans terms, fucking awesome and work really well e.g. they generalize and work in production. No one understands why. To add nuance to this, these models are awesome at interpolation, but not so much at extrapolation. Or in different terms, they generalize very well to an IID test set, but don't generalize under (even slight) distribution shift. The main reason for this is that these models tend to solve classification and regression problem quite differently from how humans do it. Broadly speaking, a large, flexible NN will find a "shortcut", i.e. a simple relation between some part of the input and the output, which may not be informative in the way we want; such as a watermark in the corner of an image, or statistical regularities in textures which disappear in slightly different lighting conditions. See e.g. https://thegradient.pub/shortcuts-neural-networks-love-to-cheat/ https://thegradient.pub/shortcuts-neural-networks-love-to-ch... I think it's fair to say that these models are great when you have an enormous dataset that covers the entire domain, but sub-Google-scale problems are usually still solved by underparametrized models (even at Google).

rich_sasha 5y ago
It depends. It really doesn’t take that much data to train a pretty stunning (if simple) RNN character-level “language model” that beats any n-gram. Or on mnist. ANNs really are a useful tool for a vast class of problems, many of which can be solved with comparatively little data. Maybe your point stands, and it’s just that some domains need less data, just saying.
- yldedly 5y ago
  >ANNs really are a useful tool for a vast class of problems, many of which can be solved with comparatively little data. For sure, it all depends on how robust the model needs to be, how strongly it needs to generalize. If your dataset covers the entire domain, you don't need a robust model. If you need strong generalization, then you need to build in stronger priors. Take f(x) = x^2. If your model only needs to work in finite interval, you just need a decent sample that covers that interval. But if it needs to generalize outside that interval, no amount of parameters will give you good performance. Outside the boundaries of the interval, the NN will either be constant (with a sigmoid activation) or linear (with ReLU type activations).

andrewnc 5y ago

Super pedantic comment. GPT-3 has 175 Billion parameters. GPT-2 was the 1.5 Billion model.

988747 5y ago

> In practice, these huge models are, in laymans terms, fucking awesome and work really well e.g. they generalize and work in production. No one understands why. How about the resulting weights? If most of them are close to 0, then that would mean that a part of the training is for NN to learn which of 1.5B parameters are relevant, and which are not.

rich_sasha 5y ago
There is something called the golden ticket theory (maybe mentioned in the paper, I’m on my phone), that says indeed that the large models are effectively ensembles of massive random models, and the top levels of the network pick the one or two that randomly happen to work. Maybe true but even then only part of the story, kernels in CNN genuinely seem to learn features like edges and textures.
talolard 5y ago
There are two answers to this. First, empirically we see that the more parameters we add the better the model performs ==> Weights continue to contribute (and aren't dead) . Second, there is a very popular paper called "The lottery ticket hypothesis" [1] that in any network you can find subnetworks that work just as well. e.g. The parameters are redundant. This was written in 2018, which is a long time ago in big NN world, so I'm not sure how it holds up to current insanity sized models. [1]https://arxiv.org/abs/1803.03635 https://arxiv.org/abs/1803.03635
- sdenton4 5y ago
  A couple notes... 1) Imagine the loss surface of a given model architecture; each point on the surface corresponds to a full set of weights, and the value at the point is the model loss. So, a billion-dimensional surface, give or take. There's a massive amount of flexibility in that space. Some models in the surface are sparse, but they are adjacent to models which are just as good but not sparse at all. Likewise, if you 'rotate' a sparse model, you can end up with an entirely equivalent dense model. So, you really need additional 'pressure' on the learning problem to ensure you actually get sparsity, even if the sparsity is in some sense natural. 2) IIUC, lottery ticket kinda breaks with larger models/problems. For small enough problems, the initial random projection given by the random starting weights is already good enough to build on. For bigger + more complicated problems, you need to really adapt in early training, and so lottery ticket breaks down.

deleted 5y ago

[deleted]