6 ms·

A New Lens on Understanding Generalization in Deep Learning

91 points by jonbaer 6y ago

joe_the_user 6y ago
To me, the strange thing about talk of "real world generalization" is that the characteristics of this world aren't described. In fact, nothing really delineates what it is - not that we humans don't have an immediate understanding of it but still. Overall, one idea is experiments on data sets that have structures but where neural network don't generalize well. Or data sets where neural generalize even better than "the real world".
- webmaven 6y ago
  > To me, the strange thing about talk of "real world generalization" is that the characteristics of this world aren't described. In fact, nothing really delineates what it is - not that we humans don't have an immediate understanding of it but still. But it is defined: In the real world, you have a finite dataset, and so have to use it over and over during training. You can shuffle the order, augment it with transformations, divide it into batches, but ultimately you reuse the whole dataset (other than the subset withheld for evaluation) in each training epoch. If the trained network deals correctly with never-before-seen examples (whether they were examples witheld from the training set or brand new ones), then it has generalized well. Ideally, you wouldn't have to go through any of that rigmarole with a finite dataset, every sample you train on would be fresh and unique, and you would have an endless stream of them. That's the 'ideal world' scenario. The same standard for generalization applies, except that every training sample is never-before-seen as well (so, no epochs).
  - joe_the_user 6y ago
    But it is defined: In the real world, you have a finite dataset, and so have to use it over and over during training. You can shuffle the order, augment it with transformations, divide it into batches, but ultimately you reuse the whole dataset (other than the subset withheld for evaluation) in each training epoch. I have no idea how "you have a finite dataset" defines "real world generalization". This line of reasoning seems completely incoherent.
    - webmaven 6y ago
      Sorry, that got scrambled in an edit due to my conflation of "real world generalization" and "real world vs. ideal world training." The result is, indeed, incoherent. Anyway, while "real world vs. ideal world training" is defined, you are correct that generalization is not, at least here. But definitions do exist, and more or less conform to what I wrote. Withholding a subset of training data for validation is meant as a proxy for the entirely new samples that will (presumably) be encountered upon deploying the model, but if the training data is biased or otherwise not representative of the real world data, then it is probable that testing the withheld data won't reveal the problem. In other words, the model generalizes well within the parameters of the training set, but not to the real-world data that has a different distribution, because the training set wasn't representative. There is some interesting discussion of this within Gwern's investigation of the apocryphal 'tank story': https://www.gwern.net/Tanks https://www.gwern.net/Tanks
alpineidyll3 6y ago
Wow, slow quarter huh?
woopwoop 6y ago
I agree with the philosophical conclusions here (e.g. I take their point on data augmentation to basically be that the name "data augmentation" is well chosen - it enlarges the restricted real world). I'm confused by the setup though. The training algorithm seeks to minimize the training loss on, in one case, the CIFAR-10 distribution D, and in the second case on D', their "ideal world" distribution. They find that these two tasks look very similar. I don't see what conclusions you can draw from this except that they did a good job training their generative model and D' is indeed very close to D, and so minimizers for the training loss on D' will look similar to those for D. (Close can mean something like the distributions D|_{label = c} and D|_{label=c'} are close in Earth-mover distance in some appropriate embedding space).
- antipaul 6y ago
  Indeed. Isn’t that the whole point of the 100 years or whatever of statistical inference, where D is a sample and is representative of D’?
plaidfuji 6y ago
I guess this makes sense, intuitively. If I understand a cat as a “furry animal with four legs, a long tail and pointy ears”, that’s a pretty generalizable model. If I see a new picture of a cat that has orange fur for the first time ever, I will still guess it’s a cat. I’ll also be able to quickly update my model parameters to include orange fur (I.e. on-line learning is fast). Whereas if my model of a cat is “brown stripy thing that’s usually on a window sill”, and I see a picture of an orange cat outdoors, not only will I probably guess it’s a pumpkin or something, it will also take many steps in parameter space to adjust my model to incorporate this new data (on-line learning will be slow).
chroem- 6y ago
> We trained a generative model on CIFAR-10, which we then used to generate ~6 million images. The scale of the dataset was chosen to ensure that it is “virtually infinite” from the model’s perspective, so that the model never resamples the same data. I don't think so, Tim. They observe identical performance between cifar-10 and cifar-5m, because the generative model for cifar-5m learned to replicate the distribution that cifar-10 was sampled from. It's the same dataset.
- kingsleyopara 6y ago
  This stood out to me too and puts a big question mark over the whole framework.
- dontreact 6y ago
  Seriously... What's surprising is that Google has an enormous dataset called JFT where they could have tested this without this confound. Just shrink the images and you can make something cifar-like and something cifar-5m-like
  - choppaface 6y ago
    or even just an ablation on Imagenet or some other “large” dataset. Did this paper get accepted because it has five pages of citations and invokes Vapnik in the second sentence?
    - preetum 6y ago
      We have ImageNet experiments in Section 4 of the full paper: https://arxiv.org/abs/2010.08127 https://arxiv.org/abs/2010.08127
      - choppaface 6y ago
        You're not ablating anything there. What happens when Train Infinity (Train 150K) doubles in size (to Train Infinity_2 -> 300K)? What happens when you add an unseen class? These are real-world conditions that hamper existing theoretical estimation of the generalization gap-- the "ideal world" always gets larger. In Bengio's group paper (Predicting the Generalization Gap https://arxiv.org/pdf/1810.00113.pdf https://arxiv.org/pdf/1810.00113.pdf ) they actually do these sorts of ablations. Also, you use K (thousands) and $K$ (latex K) interchangeably; it's really hard to decipher is K is a variable or what you mean.
deleted 6y ago
[deleted]
Haarisgrey 6y ago
Of course we got people like this!. I smoke and I’ve been a smoker for years. Most of the time I am surrounded by people who do not smoke at all. Doesn’t amaze me. I have some close friends, we hangout together international;y and vacation parties but they both have never smoked weed and I am a regular smoker and this thing doesn’t affect our friendship at all. It’s not about something being legal or illegal. If a thing is facilitating you either way it is totally okay for the other person to use it. I started using it for my chronic pain and eventually ended up like this. https://www.weedmarket420.us/ https://www.weedmarket420.us/ Being a marijuana enthusiast I keep on researching for good material and here I came across 5 new researches! Have a good day!
davnn 6y ago
So the main claim of this paper is: "...good models and training procedures are those that (1) optimize quickly in the ideal world and (2) do not optimize too quickly in the real world." Will be interesting to replicate those results on different data.
ersiees 6y ago
The real comparison would have been between Cifar-10 (or Imagenet) and a downsampled version there of, not upsampled, and I actually know for sure that this harms performance.. So this ideal world and real world training are definitely not the same!
- preetum 6y ago
  This is the exact comparison we make in the paper. We have subsampled ImageNet experiments as well; see Figure 3 in the full paper: https://arxiv.org/abs/2010.08127 https://arxiv.org/abs/2010.08127
sgt101 6y ago
Resnet works better than MLP not because it optimizes faster, but because the architecture of the network contains a bias towards spatially structured data. MLP's see images as sorted or scrambled, the relationship of each pixel to another pixel is the same at the start (or randomly stronger or weaker if you initialize the network randomly).
blueblisters 6y ago
Maybe I'm pedantic but the choice of the terms "real-world" and "ideal world" are really confusing for identifying a finite vs. infinite training regime. Even in the real-world, you can have effectively infinite training sample if you have a large enough dataset, just like they did for the experiments.
mustafa_pasi 6y ago
Is there anyone doing Deep Learning research who acknowledges that it is an optimization subfield and accordingly uses rigorous math? I don't mean to be snarky, and I am sure there is some use for this result, but I hope somebody is working on the math and I want to know who that person is.
- notretarded 6y ago
  No, it doesn't pay the bills that way
- plaidfuji 6y ago
  I like to imagine that it’s just one guy named Adam.

91 points by jonbaer 6y ago

To me, the strange thing about talk of "real world generalization" is that the characteristics of this world aren't described. In fact, nothing really delineates what it is - not that we humans don't have an immediate understanding of it but still. Overall, one idea is experiments on data sets that have structures but where neural network don't generalize well. Or data sets where neural generalize even better than "the real world".

webmaven 6y ago
> To me, the strange thing about talk of "real world generalization" is that the characteristics of this world aren't described. In fact, nothing really delineates what it is - not that we humans don't have an immediate understanding of it but still. But it is defined: In the real world, you have a finite dataset, and so have to use it over and over during training. You can shuffle the order, augment it with transformations, divide it into batches, but ultimately you reuse the whole dataset (other than the subset withheld for evaluation) in each training epoch. If the trained network deals correctly with never-before-seen examples (whether they were examples witheld from the training set or brand new ones), then it has generalized well. Ideally, you wouldn't have to go through any of that rigmarole with a finite dataset, every sample you train on would be fresh and unique, and you would have an endless stream of them. That's the 'ideal world' scenario. The same standard for generalization applies, except that every training sample is never-before-seen as well (so, no epochs).
- joe_the_user 6y ago
  But it is defined: In the real world, you have a finite dataset, and so have to use it over and over during training. You can shuffle the order, augment it with transformations, divide it into batches, but ultimately you reuse the whole dataset (other than the subset withheld for evaluation) in each training epoch. I have no idea how "you have a finite dataset" defines "real world generalization". This line of reasoning seems completely incoherent.
  - webmaven 6y ago
    Sorry, that got scrambled in an edit due to my conflation of "real world generalization" and "real world vs. ideal world training." The result is, indeed, incoherent. Anyway, while "real world vs. ideal world training" is defined, you are correct that generalization is not, at least here. But definitions do exist, and more or less conform to what I wrote. Withholding a subset of training data for validation is meant as a proxy for the entirely new samples that will (presumably) be encountered upon deploying the model, but if the training data is biased or otherwise not representative of the real world data, then it is probable that testing the withheld data won't reveal the problem. In other words, the model generalizes well within the parameters of the training set, but not to the real-world data that has a different distribution, because the training set wasn't representative. There is some interesting discussion of this within Gwern's investigation of the apocryphal 'tank story': https://www.gwern.net/Tanks https://www.gwern.net/Tanks

alpineidyll3 6y ago

Wow, slow quarter huh?

woopwoop 6y ago

I agree with the philosophical conclusions here (e.g. I take their point on data augmentation to basically be that the name "data augmentation" is well chosen - it enlarges the restricted real world). I'm confused by the setup though. The training algorithm seeks to minimize the training loss on, in one case, the CIFAR-10 distribution D, and in the second case on D', their "ideal world" distribution. They find that these two tasks look very similar. I don't see what conclusions you can draw from this except that they did a good job training their generative model and D' is indeed very close to D, and so minimizers for the training loss on D' will look similar to those for D. (Close can mean something like the distributions D|_{label = c} and D|_{label=c'} are close in Earth-mover distance in some appropriate embedding space).

antipaul 6y ago
Indeed. Isn’t that the whole point of the 100 years or whatever of statistical inference, where D is a sample and is representative of D’?

plaidfuji 6y ago

I guess this makes sense, intuitively. If I understand a cat as a “furry animal with four legs, a long tail and pointy ears”, that’s a pretty generalizable model. If I see a new picture of a cat that has orange fur for the first time ever, I will still guess it’s a cat. I’ll also be able to quickly update my model parameters to include orange fur (I.e. on-line learning is fast). Whereas if my model of a cat is “brown stripy thing that’s usually on a window sill”, and I see a picture of an orange cat outdoors, not only will I probably guess it’s a pumpkin or something, it will also take many steps in parameter space to adjust my model to incorporate this new data (on-line learning will be slow).

chroem- 6y ago

> We trained a generative model on CIFAR-10, which we then used to generate ~6 million images. The scale of the dataset was chosen to ensure that it is “virtually infinite” from the model’s perspective, so that the model never resamples the same data. I don't think so, Tim. They observe identical performance between cifar-10 and cifar-5m, because the generative model for cifar-5m learned to replicate the distribution that cifar-10 was sampled from. It's the same dataset.

kingsleyopara 6y ago
This stood out to me too and puts a big question mark over the whole framework.
dontreact 6y ago
Seriously... What's surprising is that Google has an enormous dataset called JFT where they could have tested this without this confound. Just shrink the images and you can make something cifar-like and something cifar-5m-like
- choppaface 6y ago
  or even just an ablation on Imagenet or some other “large” dataset. Did this paper get accepted because it has five pages of citations and invokes Vapnik in the second sentence?
  - preetum 6y ago
    We have ImageNet experiments in Section 4 of the full paper: https://arxiv.org/abs/2010.08127 https://arxiv.org/abs/2010.08127
    - choppaface 6y ago
      You're not ablating anything there. What happens when Train Infinity (Train 150K) doubles in size (to Train Infinity_2 -> 300K)? What happens when you add an unseen class? These are real-world conditions that hamper existing theoretical estimation of the generalization gap-- the "ideal world" always gets larger. In Bengio's group paper (Predicting the Generalization Gap https://arxiv.org/pdf/1810.00113.pdf https://arxiv.org/pdf/1810.00113.pdf ) they actually do these sorts of ablations. Also, you use K (thousands) and $K$ (latex K) interchangeably; it's really hard to decipher is K is a variable or what you mean.

deleted 6y ago

[deleted]

Haarisgrey 6y ago

Of course we got people like this!. I smoke and I’ve been a smoker for years. Most of the time I am surrounded by people who do not smoke at all. Doesn’t amaze me. I have some close friends, we hangout together international;y and vacation parties but they both have never smoked weed and I am a regular smoker and this thing doesn’t affect our friendship at all. It’s not about something being legal or illegal. If a thing is facilitating you either way it is totally okay for the other person to use it. I started using it for my chronic pain and eventually ended up like this. https://www.weedmarket420.us/ https://www.weedmarket420.us/ Being a marijuana enthusiast I keep on researching for good material and here I came across 5 new researches! Have a good day!

davnn 6y ago

So the main claim of this paper is: "...good models and training procedures are those that (1) optimize quickly in the ideal world and (2) do not optimize too quickly in the real world." Will be interesting to replicate those results on different data.

ersiees 6y ago

The real comparison would have been between Cifar-10 (or Imagenet) and a downsampled version there of, not upsampled, and I actually know for sure that this harms performance.. So this ideal world and real world training are definitely not the same!

preetum 6y ago
This is the exact comparison we make in the paper. We have subsampled ImageNet experiments as well; see Figure 3 in the full paper: https://arxiv.org/abs/2010.08127 https://arxiv.org/abs/2010.08127

sgt101 6y ago

Resnet works better than MLP not because it optimizes faster, but because the architecture of the network contains a bias towards spatially structured data. MLP's see images as sorted or scrambled, the relationship of each pixel to another pixel is the same at the start (or randomly stronger or weaker if you initialize the network randomly).

blueblisters 6y ago

Maybe I'm pedantic but the choice of the terms "real-world" and "ideal world" are really confusing for identifying a finite vs. infinite training regime. Even in the real-world, you can have effectively infinite training sample if you have a large enough dataset, just like they did for the experiments.

mustafa_pasi 6y ago

Is there anyone doing Deep Learning research who acknowledges that it is an optimization subfield and accordingly uses rigorous math? I don't mean to be snarky, and I am sure there is some use for this result, but I hope somebody is working on the math and I want to know who that person is.

notretarded 6y ago
No, it doesn't pay the bills that way
plaidfuji 6y ago
I like to imagine that it’s just one guy named Adam.