8 ms·

A new link to an old model could crack the mystery of deep learning

173 points by the__prestige 5y ago

flerovium 5y ago
Why do I keep seeeing quanta magazine on hackernews? They've published a lot of very speculative results, often with simple inaccuracies. Can I filter this out?
shenberg 5y ago
Microsoft research has a really nice paper showing that the NTK explanation is very unlikely: https://www.microsoft.com/en-us/research/blog/three-mysteries-in-deep-learning-ensemble-knowledge-distillation-and-self-distillation/ https://www.microsoft.com/en-us/research/blog/three-mysterie...
jonnycomputer 5y ago
Incidentally, yesterday I submitted a wordpress blog post about some of these issues. https://news.ycombinator.com/item?id=28817666 https://news.ycombinator.com/item?id=28817666 https://matloff.wordpress.com/2020/11/11/the-notion-of-double-descent/ https://matloff.wordpress.com/2020/11/11/the-notion-of-doubl...
erostrate 5y ago
A lot of effort went into these efforts to understand neural networks in terms or kernels or SVMs. To the best of my knowledge, these efforts have not inspired useful new architectures, nor have they made useful experimental predictions on real neural nets, nor have they had any significant impact on important machine learning benchmarks. I think some researchers are refusing to accept the idea that machine learning is very much an experimental science today, and the (very cool) mathematics of kernels, SVMs, empirical risk minimization, bayesian statistics, etc. are simply no longer useful in the large scale regime. My prediction is that there will be a useful "deep learning theory" in the future, but it will look a lot more like physics (such as Kaplan's scaling laws) than early 21st century machine learning mathematics/statistics.
- curiousgal 5y ago
  This is ironic because the exact same thing could have been said about Neural Networks back in the 90s. Just because there aren't any practical applications now of "mathy ML", it does not mean it's a dead end.
  - civilized 5y ago
    I remember how uncool neural nets were back then! Kernel SVM was so hot, but LeCun said it was nothing more than template matching with fancy math. And he was right...
- ThereIsNoWorry 5y ago
  Neural networks are very non-linear. The blessing and curse of non-linearity is its usefulness while escaping what mathematical logic inherently is able to model. So yes, I strongly assume there will never be a mathematically sound theory that predicts deep learning in any meaningful way. Same problem with everything non-linear. From two-body to chaotic systems, biological interactions to the stock market. All just wishful thinking until Mathematicians give up and instead continue to play around in their well-defined esoteric spaces.
  - gmadsen 5y ago
    you don't seem to have a background in mathematics. A huge(and growing) body of work exists for every example you gave. Obviously mathematicians are interested in nonlinear systems
  - goatlover 5y ago
    So you're saying Hari Seldon's psychohistory is BS?
  - ravi-delia 5y ago
    I think you conflate several ideas of understanding here. We don't have good ways of predicting chaotic systems, but we often know a lot about them. Just knowing they're highly sensitive would be hard to confirm without math.
abetusk 5y ago
I've only skimmed the article, but the idea looks to be "Kernel methods" [0] (and showing their equivalence in the infinite middle layer width limit?). If anyone has more insight into the actual idea, I would appreciate an explanation. In the article, there are two advertisements to older articles. One is "New Theory Cracks Open the Black Box of Deep Learning" from 2017 and the other is "Foundations Built for a General Theory of Neural Networks" from 2019. [0] https://www.quantamagazine.org/a-new-link-to-an-old-model-could-crack-the-mystery-of-deep-learning-20211011/ https://www.quantamagazine.org/a-new-link-to-an-old-model-co...
- leecarraher 5y ago
  They also have an embedded link to this paper(https://arxiv.org/abs/1810.02054 https://arxiv.org/abs/1810.02054 ) regarding the provable convergence of neural networks by non-convex gradient descent optimization. You are correct, the researchers do no propose an alternative to sgd based training (or at least according to the article). It did seem like it was going toward an alternative training method, perhaps SMO on kernel SVM, but then it pivots to describing the theoretical result for infinte kernel svm from the linked article.
homerowilson 5y ago
A nice, short, recent paper along these lines (not mentioned in the article) is "Every Model Learned by Gradient Descent Is Approximately a Kernel Machine": https://arxiv.org/abs/2012.00152 https://arxiv.org/abs/2012.00152
r-zip 5y ago
I'd recommend anyone interested in the NTK and its limitations to check out Greg Yang's work [1] and the recent book by Roberts, Yaida, and Hanin [2]. There's clearly a lot more work to be done on neural network scaling limits; the NTK is overly simplistic (though useful). 1: https://www.microsoft.com/en-us/research/people/gregyang/ https://www.microsoft.com/en-us/research/people/gregyang/ 2: https://arxiv.org/abs/2106.10165 https://arxiv.org/abs/2106.10165
- scythmic_waves 5y ago
  Agreed, Greg Yang's work is really interesting. Here's a reddit thread by Yang discussing his research into why NNs are not Kernel Machines [1]. [1] https://www.reddit.com/r/MachineLearning/comments/k8h01q/r_wide_neural_networks_are_feature_learners_not/ https://www.reddit.com/r/MachineLearning/comments/k8h01q/r_w...
mark_l_watson 5y ago
Is there much mystery? SGD in a small number of dimensions is easy enough to understand, and as Hinton said in an online class I took years ago: to visualize very high numbers in dimensions, close your eyes and shout out the number. He was joking but his comment convinced me to disregard the concept of millions of billions of dimensions and instead concentrate on practical techniques that enable training many layers, architectures that are multi-headed, etc., and use model architectures that are known to work for different types of problems.
- r-zip 5y ago
  That's a fine attitude for applied ML, but aren't you even a little bit interested in why these things work so well? It's not obvious.
  - datameta 5y ago
    It's fascinating how embedding informational complexity in higher order relationships somehow gives rise to a map of represented concepts that are extendable with new data. Explainable NN models are going to be an absolute necessity - such as in the justice system or if we employ them in autonomous driving. Tangentially, I don't believe GPT-N will be the way to achieve AGI, however we can learn much on said road travelled.
    - yomama99 5y ago
      > or if we employ them in autonomous driving. This is not looking to be the case right now.. Companies are doing quite well without explainable models. SDC companies like Waymo are hoping it will be sufficient to: 1. Produce a low failure rate estimate, based on millions of virtual miles/fake cities. 2. Combine several NN outputs in a logical way. That way there are at least some interpretable values that lead to the result. e.g. a separate model that produces candidate trajectories of surrounding vehicles, one to perceive sensor input, one to plan, etc.
- JustFinishedBSG 5y ago
  > Is there much mystery? Yes ? > SGD in a small number of dimensions is easy enough to understand Is it?
dekhn 5y ago
I remember being introduced to SVMs in the context of machine learning in biology before deep neural networks: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2559884/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2559884/ I left the ML community in disgust at how unsavory these ML methods were.
- ryan93 5y ago
  I mean you are gonna have to expand on that. How can you be disgusted by some simple math?
  - dekhn 5y ago
    I dunno why I find the idea of "finding the hyperplane that maximally separates classes" so unpleasant compared to the approach that makes sense to me "optimize the weights of a parameterized function to minimize an objective function". I imagine under the hood they're not really different. Generally, I think SVMs appear to statisticians while deep learning appeals to physicists and probabilicians.
ppod 5y ago
The explanations in quanta magazine are so good. They are more in depth than a usual pop-sci article, but very followable.
mvcalder 5y ago
When I first read about the NTK I was motivated to try an experiment to see if the kernel depended on where in the parameter space one evaluated it. I was surprised to see in my simple example the answer was "no". Here is a write up I did: https://arxiv.org/pdf/2104.05874.pdf https://arxiv.org/pdf/2104.05874.pdf
charleshmartin 5y ago
Here's an alternative approach, that actually provides real world results https://calculatedcontent.com/2019/12/03/towards-a-new-theory-of-learning-statistical-mechanics-of-deep-neural-networks/ https://calculatedcontent.com/2019/12/03/towards-a-new-theor... Using techniques from statistical mechanics and strongly correlated systems, we can compute the average-case-behavior of a real world DNN We believe we can reproduce some of the results of the NTK by using a Gaussian Random Matrix. But if we use a more realistic, heavy tailed matrix, we get more practical results A early fork of the theory has been published in JMLR https://arxiv.org/abs/1810.01075 https://arxiv.org/abs/1810.01075 and the empirical results in Nature Communications https://www.nature.com/articles/s41467-021-24025-8 https://www.nature.com/articles/s41467-021-24025-8 and we have an open source tool , weightwatcher, which can be used in production pip install weightwatcher https://github.com/CalculatedContent/WeightWatcher https://github.com/CalculatedContent/WeightWatcher Please give it a try and let me know if it is useful to you
- thisnormaluser 5y ago
  Nature Communications is not Nature. But whatever.
  - chaxor 5y ago
    I actually have a preference towards Nature Communications over Nature, as Nature Communications actually allows more pages for explanation and as such typically has more rigorous research. On the flip side, Nature often has bolder claims with less convincing evidence, which often reads similar to pop-sci articles.
  - deleted 5y ago
    [deleted]
  - deleted 5y ago
    [deleted]
  - charleshmartin 5y ago
    That's fine. Here's a few of my online talks: UC Berkeley / ICSI: https://www.youtube.com/watch?v=6Zgul4oygMc https://www.youtube.com/watch?v=6Zgul4oygMc Stanford ICME: https://www.youtube.com/watch?v=PQUItQi-B-I https://www.youtube.com/watch?v=PQUItQi-B-I and a couple of Mikes's Institute for Pure & Applied Mathematics (IPAM) : https://www.youtube.com/watch?v=fmVuNRKsQa8 https://www.youtube.com/watch?v=fmVuNRKsQa8 Physics Informed Machine Learning: https://www.youtube.com/watch?v=eXhwLtjtUsI https://www.youtube.com/watch?v=eXhwLtjtUsI and our KDD workshop KDD: https://dl.acm.org/doi/abs/10.1145/3292500.3332294 https://dl.acm.org/doi/abs/10.1145/3292500.3332294 many more talks and papers available And my favorite, a podcast that has featured LeCun himself: https://blog.rebellionresearch.com/blog/theoretical-physicist-dr-charles-martin-on-deep-learning https://blog.rebellionresearch.com/blog/theoretical-physicis...
randcraw 5y ago
The article is intriguing, but is the objective of the mathematicians of more than theoretical value? It seems to me that proving DNNs to be equivalent to SVM kernels won't improve them appreciably -- they won't become more accurate or precise, not faster, nor more economical. More importantly, the achilles heel of DNNs won't be fixed -- their inability to explain their reasoning or annotate their inner workings to enable tuning or repair. Replacing black box linear algebra with black box math does not seem like much of a step forward.
- charleshmartin 5y ago
  agreed!
- JustFinishedBSG 5y ago
  > is the objective of the mathematicians of more than theoretical value? Why should it have any other value?
light_hue_1 5y ago
> By all accounts, deep neural networks like VGG have way too many parameters and should overfit. But they don’t. This isn't true at all. You can only make this statement relative to a dataset. And VGG is almost always trained on ImageNet. A standard VGG16 has 138 million parameters. ImageNet has 1 million images each is 244x244 (~50k). There are ~431 input data points per parameter in VGG16. VGG is not overparameterized for ImageNet. There are plenty of network/dataset pairs they could have given as an example, but this isn't it.
- marcinzm 5y ago
  VGG also has regularization (L2, dropout, maxpool) if I remember correctly so it's sort of odd to use that as an example of overfitting when it explicitly tunes against that. I'm guessing the epochs and learning rate were also tuned to lower overfitting. The same is true to every other popular network out there.
- derbOac 5y ago
  I have so many questions about this now. For example, how often are people not distinguishing between amount of data versus number of cases? Do we really have any sense of how "effective number of parameters" scales with network features? This article had me wondering if peoples' understanding of these things is backward or something.
  - light_hue_1 5y ago
    > I have so many questions about this now. For example, how often are people not distinguishing between amount of data versus number of cases? To state your question more broadly. Data has structure that's why we can learn in the first place. Can we separately talk about the structure vs number of examples of the different structures in our data? In computer vision this would be sort of like talking about how much information is within an object class vs between object class. We don't know. And we don't have the mathematical tools to talk about this today. > Do we really have any sense of how "effective number of parameters" scales with network features? This article had me wondering if peoples' understanding of these things is backward or something. It's not backward, it's non-existent. We basically try things and when we have too many parameters we see that performance isn't increasing (or isn't increasing enough to be worth it, or with unsupervised methods and massive datasets we just max out the size of network we can train).
qPM9l3XJrF 5y ago
>By all accounts, deep neural networks like VGG have way too many parameters and should overfit. I keep seeing this claim pop up but I've never seen a citation to support it. My understanding is that e.g. statistical learning theory lets you prove theorems of the form "IF it has this number of parameters or fewer, THEN it won't overfit". First year logic is enough to know this theorem tells you nothing in the scenario where the model has lots of parameters. Can anyone provide a solid citation to support the claim that models with many parameters "should" overfit? BTW, another example of a model with lots of parameters which doesn't overfit is random forests / gradient boosting. I'm wondering if this "should overfit" claim is less of a global property of machine learning models in general, and more just a property of the particular models that this set of researchers have most familiarity with (SVMs).
- JustFinishedBSG 5y ago
  Yes usual results in statistical learning theory are upper bound on the generalization error, so if you have an "overparametrized" (number of parameters is actually a useless metric but that's another discussion altogether) then the best you can say is "well I expect the error to be at most A FREAKING LOT", which is not a very interesting result.
JustFinishedBSG 5y ago
I find the majority of the comments in this thread pretty concerning and quite honestly shocking. I didn't expect HN to be basically so obscurantist and to act so similarly to medieval alchemists (and that's not being fair to alchemists). So apparently theory is useless, all that matters is the "art" and empirical results ? Especially since quite a few of the comments here dismiss this work, and related, with a quick "well it's trivial/ well known?" and then procede to give their own personal explanation that show a clear misunderstanding of the topic.
- quietbritishjim 5y ago
  I don't have any skin in this game, but it's clear even to me that the comparison with alchemists isn't valid: neural nets, and CNNs in particular have achieved actual results, whereas alchemists did not.
sgt101 5y ago
By the time I finished my Ph.D. in 1997 I was stone cold certain that SVM's were equivalent to NN's, and that 3 layers could do anything. I saw Y.Lecuns work on handwriting and thought it an oddity. I gave up on machine learning as I thought it was all done and concentrated on applications. In 2011 I was staggered to realise what a load of bollocks I had been bongoogled into believing in, and annoyed. I am never going to fall for the "if we assume x, and y and only talk about this bit of the system then it's all the same" line again. It may be mathematically nice to prove things with simplifying assumptions, but the ICML community basically walked up a blind ally because of this desire for rigor and elegance in preference to reality. ML is not mathematics, because it's tied to data, and data is a product of nature. The study of nature is Science. This is the difference between String Theory and Physics, and it's the difference between Computer Science and ML as well.
- jhgb 5y ago
  Isn't it still the case mathematically that 3 layers can do theoretically anything? Which of course may be a different claim that they can do that practically.
  - amelius 5y ago
    > Isn't it still the case mathematically that 3 layers can do theoretically anything? What do you (and parent) mean by "do"? Inference, or training, or both?
    - sgt101 5y ago
      The point is that a three layer network can approximate any function, and back propagation can be used to train it to do so. You could train a 3 layer net to do imagenet, I think it was Minsky that showed the first bit of that in "Perceptrons", and Hinton showed it in his backprop paper with Rumelheart (?) but I can't be arsed to look it up, someone will pop up and show that it was actually Leonardo Da Vinci or that German fella who has apparently invented everything ever and never tires of telling everyone about it no matter how hard we all ignore him. However, clever people (Hinton, Lecun, others v.smart and determined and all should get a lot of credit) invented some tricks to train networks faster (and carefully showed that they really worked, despite what people said about them, especially people at ICML), and computers got many faster, and it became obvious (which it was to anyone who both could do the sums and had the imagination to think about it - which I didn't, at all, sad for me) that broad networks are much much slower to train than deep networks for an equivalent number of neurons (if you use tricks like regularization and dropout) and this makes large deep networks feasible. I did play with deep networks in like 1994, because I wrote a C program to let me write neural networks - and I could do anything I liked with it on my Sun workstation and my supervisors had no idea about anything. However, I set this "deep" network off training to see what would happen and went to the pub. When I came back I interrupted it and looked at the model file to figure out how much the parameters had changed since initialization (I didn't have any of the good small value initialization stuff, I think I set them to 0) and they had changed so little that I did some sums and decided that the model would not be finished training before I finished grad school if I finished at all. So, I went off and built various types of booster on an MPI network that most of my colleagues didn't realize was running on their nice Sun workstations instead. Talking to other people in pubs at conferences has since convinced me that most of the grad students working on ML in the 90's did the same thing.
YeGoblynQueenne 5y ago
The article leaves "generalisation" without further explanation and as usual, there's a lot of confusion about what, exactly, is meant by the claim that overparameterised deep neural networks "generalize astoundingly well to new data". So, what is meant is that such networks (let's call them complex networks for short) generalise well to test data. But what exactly is "test data"? Here's how it works. A team has some data, let's call it D. The team partitions the data to a training and test set, T₁ and T₂. The team trains their system on T₁ and tests it on T₂. To choose the model to test on T₂, the team may perform cross-validation which further partitions T₁ to k validation partitions, T₁₁, ..., T₁ₖ. In the most common cross-validation scheme, k-fold cross-validation, the team trains on k-1 validation partitions of T₁ and tests on one of the k validation partitions, trying all combinations of k-1 training and 1 testing, partitions _of T₁_ (because we're still in validation, which can be very confusing). At the end of this process the team have k models each of which is trained on k-1 different partitions, and tested on one of k different partitions, of the training set T₁. The team chooses the model with the best accuracy (or whatever the metric they're measuring) and then they test this model on T₂, the testing partition. Finally, the team report the accuracy (etc) of their trained model on the testing partition, T₂, and basically claim (though generally the claim is implicit) that the accuracy of their best model on T₂ is a more or less accurate estimate of the accuracy of the model on unseen data, in the real world, i.e. data not included in D, either as T₁ or T₂. Such truly unseen data (it was not available _to the team_ at training time) is sometimes referred to as "out of distribution" data and let's call it that for simplicity [1]. So, with this background at hand, we now understand that when the article above (and neural net researchers) say that complex networks "generalise well", they mean "on test data". Which is mildly surprising given that various assumptions make it less surprising that they'd do well on training data etc etc. There are two key observations to make here. One is that complex networks generalise well on test data _when the researchers have access to the test data_. When researchers have access to test data, the training regime I outline above includes a further step: the team looks at the accuracy of their system on test data, find it to be abysmal, and, crestfallen, abandon their expensive research and start completely from scratch, because obviously they wouldn't just try to tweak their system to perform better on the test data! That would be essentially peeking at the test data and guiding their system to perform well on it (e.g. by tuning hyperparameters or by better "random" initialisation)! I'm kidding. Of course a team who finds their system performs badly on test data will tweak their system to perform better on the test data. They'll peek. And peek again. Not only they'll peek, they'll conduct an automated hyperparameter-tuning search (a "grid search") _on the test data_! That is what's known in science, I believe, as a "fishing expedition". Two, in the typical training regime, the training partition T₁ is 80% of D, the entire dataset, and T₂ is 20% of D. And when T₁ is partitioned for cross-validation, the validation-training partition (the k-1 folds trained on) is also usually 4 times larger than the validation-testing partition (the single fold tested on). Why? Because complex networks need Big Data to train on. But, what happens when you train on 80% of your data and test on 20% of it? What happens is that your estimates of accuracy are not very good estimates, because, even if your data is Really Big, 20% of it is likely to miss a big chunk of the variation found in the 80% of the data, and so you're basically only estimating your system's accuracy on a small sub-set of the features that it needs to represent to be said to "generalise well". For this reason, complex networks, like all Big Data approaches, are pretty much crap at generalising to "O.O.D." data, or, more to the point, estimates of their accuracy on OOD data are just pretty bad. In practice, deep neural net systems that have amazing performance "in the lab", can be expected to lose 20-40% of their performance "in the real world" [2]. Bottom line, there's no big mystery why complex networks "generalise" so well and there is no need to seek an explanation for this in kernel machines. Complex networks generalise so well because the kind of generalisation they're so good at is the kind of generalisation achieved by humans tweaking the network until it overfits _to the test data_. And that's the worst kind of overfitting. __________ [1] Under PAC-Learning assumptions we can expect the performance of any machine learning system on data that is truly "out of distribution", in the sense that it is drawn from a distribution radically different than the distribution of the training data, to be really bad, because we assume distributional consistency between training and "true" data. But that's a bit of a quibble, and "OOD" has established itself as a half-understood jargon term so I let this rest. [2] I had a reference for that somewhere... can't find it. You'll have to take my word for it. Would I lie to you?

173 points by the__prestige 5y ago

Why do I keep seeeing quanta magazine on hackernews? They've published a lot of very speculative results, often with simple inaccuracies. Can I filter this out?

shenberg 5y ago

Microsoft research has a really nice paper showing that the NTK explanation is very unlikely: https://www.microsoft.com/en-us/research/blog/three-mysteries-in-deep-learning-ensemble-knowledge-distillation-and-self-distillation/ https://www.microsoft.com/en-us/research/blog/three-mysterie...

jonnycomputer 5y ago

Incidentally, yesterday I submitted a wordpress blog post about some of these issues. https://news.ycombinator.com/item?id=28817666 https://news.ycombinator.com/item?id=28817666 https://matloff.wordpress.com/2020/11/11/the-notion-of-double-descent/ https://matloff.wordpress.com/2020/11/11/the-notion-of-doubl...

erostrate 5y ago

A lot of effort went into these efforts to understand neural networks in terms or kernels or SVMs. To the best of my knowledge, these efforts have not inspired useful new architectures, nor have they made useful experimental predictions on real neural nets, nor have they had any significant impact on important machine learning benchmarks. I think some researchers are refusing to accept the idea that machine learning is very much an experimental science today, and the (very cool) mathematics of kernels, SVMs, empirical risk minimization, bayesian statistics, etc. are simply no longer useful in the large scale regime. My prediction is that there will be a useful "deep learning theory" in the future, but it will look a lot more like physics (such as Kaplan's scaling laws) than early 21st century machine learning mathematics/statistics.

curiousgal 5y ago
This is ironic because the exact same thing could have been said about Neural Networks back in the 90s. Just because there aren't any practical applications now of "mathy ML", it does not mean it's a dead end.
- civilized 5y ago
  I remember how uncool neural nets were back then! Kernel SVM was so hot, but LeCun said it was nothing more than template matching with fancy math. And he was right...
ThereIsNoWorry 5y ago
Neural networks are very non-linear. The blessing and curse of non-linearity is its usefulness while escaping what mathematical logic inherently is able to model. So yes, I strongly assume there will never be a mathematically sound theory that predicts deep learning in any meaningful way. Same problem with everything non-linear. From two-body to chaotic systems, biological interactions to the stock market. All just wishful thinking until Mathematicians give up and instead continue to play around in their well-defined esoteric spaces.
- gmadsen 5y ago
  you don't seem to have a background in mathematics. A huge(and growing) body of work exists for every example you gave. Obviously mathematicians are interested in nonlinear systems
- goatlover 5y ago
  So you're saying Hari Seldon's psychohistory is BS?
- ravi-delia 5y ago
  I think you conflate several ideas of understanding here. We don't have good ways of predicting chaotic systems, but we often know a lot about them. Just knowing they're highly sensitive would be hard to confirm without math.

abetusk 5y ago

I've only skimmed the article, but the idea looks to be "Kernel methods" [0] (and showing their equivalence in the infinite middle layer width limit?). If anyone has more insight into the actual idea, I would appreciate an explanation. In the article, there are two advertisements to older articles. One is "New Theory Cracks Open the Black Box of Deep Learning" from 2017 and the other is "Foundations Built for a General Theory of Neural Networks" from 2019. [0] https://www.quantamagazine.org/a-new-link-to-an-old-model-could-crack-the-mystery-of-deep-learning-20211011/ https://www.quantamagazine.org/a-new-link-to-an-old-model-co...

leecarraher 5y ago
They also have an embedded link to this paper(https://arxiv.org/abs/1810.02054 https://arxiv.org/abs/1810.02054 ) regarding the provable convergence of neural networks by non-convex gradient descent optimization. You are correct, the researchers do no propose an alternative to sgd based training (or at least according to the article). It did seem like it was going toward an alternative training method, perhaps SMO on kernel SVM, but then it pivots to describing the theoretical result for infinte kernel svm from the linked article.

homerowilson 5y ago

A nice, short, recent paper along these lines (not mentioned in the article) is "Every Model Learned by Gradient Descent Is Approximately a Kernel Machine": https://arxiv.org/abs/2012.00152 https://arxiv.org/abs/2012.00152

r-zip 5y ago

I'd recommend anyone interested in the NTK and its limitations to check out Greg Yang's work [1] and the recent book by Roberts, Yaida, and Hanin [2]. There's clearly a lot more work to be done on neural network scaling limits; the NTK is overly simplistic (though useful). 1: https://www.microsoft.com/en-us/research/people/gregyang/ https://www.microsoft.com/en-us/research/people/gregyang/ 2: https://arxiv.org/abs/2106.10165 https://arxiv.org/abs/2106.10165

scythmic_waves 5y ago
Agreed, Greg Yang's work is really interesting. Here's a reddit thread by Yang discussing his research into why NNs are not Kernel Machines [1]. [1] https://www.reddit.com/r/MachineLearning/comments/k8h01q/r_wide_neural_networks_are_feature_learners_not/ https://www.reddit.com/r/MachineLearning/comments/k8h01q/r_w...

mark_l_watson 5y ago

Is there much mystery? SGD in a small number of dimensions is easy enough to understand, and as Hinton said in an online class I took years ago: to visualize very high numbers in dimensions, close your eyes and shout out the number. He was joking but his comment convinced me to disregard the concept of millions of billions of dimensions and instead concentrate on practical techniques that enable training many layers, architectures that are multi-headed, etc., and use model architectures that are known to work for different types of problems.

r-zip 5y ago
That's a fine attitude for applied ML, but aren't you even a little bit interested in why these things work so well? It's not obvious.
- datameta 5y ago
  It's fascinating how embedding informational complexity in higher order relationships somehow gives rise to a map of represented concepts that are extendable with new data. Explainable NN models are going to be an absolute necessity - such as in the justice system or if we employ them in autonomous driving. Tangentially, I don't believe GPT-N will be the way to achieve AGI, however we can learn much on said road travelled.
  - yomama99 5y ago
    > or if we employ them in autonomous driving. This is not looking to be the case right now.. Companies are doing quite well without explainable models. SDC companies like Waymo are hoping it will be sufficient to: 1. Produce a low failure rate estimate, based on millions of virtual miles/fake cities. 2. Combine several NN outputs in a logical way. That way there are at least some interpretable values that lead to the result. e.g. a separate model that produces candidate trajectories of surrounding vehicles, one to perceive sensor input, one to plan, etc.
JustFinishedBSG 5y ago
> Is there much mystery? Yes ? > SGD in a small number of dimensions is easy enough to understand Is it?

dekhn 5y ago

I remember being introduced to SVMs in the context of machine learning in biology before deep neural networks: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2559884/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2559884/ I left the ML community in disgust at how unsavory these ML methods were.

ryan93 5y ago
I mean you are gonna have to expand on that. How can you be disgusted by some simple math?
- dekhn 5y ago
  I dunno why I find the idea of "finding the hyperplane that maximally separates classes" so unpleasant compared to the approach that makes sense to me "optimize the weights of a parameterized function to minimize an objective function". I imagine under the hood they're not really different. Generally, I think SVMs appear to statisticians while deep learning appeals to physicists and probabilicians.

ppod 5y ago

The explanations in quanta magazine are so good. They are more in depth than a usual pop-sci article, but very followable.

mvcalder 5y ago

When I first read about the NTK I was motivated to try an experiment to see if the kernel depended on where in the parameter space one evaluated it. I was surprised to see in my simple example the answer was "no". Here is a write up I did: https://arxiv.org/pdf/2104.05874.pdf https://arxiv.org/pdf/2104.05874.pdf

charleshmartin 5y ago

Here's an alternative approach, that actually provides real world results https://calculatedcontent.com/2019/12/03/towards-a-new-theory-of-learning-statistical-mechanics-of-deep-neural-networks/ https://calculatedcontent.com/2019/12/03/towards-a-new-theor... Using techniques from statistical mechanics and strongly correlated systems, we can compute the average-case-behavior of a real world DNN We believe we can reproduce some of the results of the NTK by using a Gaussian Random Matrix. But if we use a more realistic, heavy tailed matrix, we get more practical results A early fork of the theory has been published in JMLR https://arxiv.org/abs/1810.01075 https://arxiv.org/abs/1810.01075 and the empirical results in Nature Communications https://www.nature.com/articles/s41467-021-24025-8 https://www.nature.com/articles/s41467-021-24025-8 and we have an open source tool , weightwatcher, which can be used in production pip install weightwatcher https://github.com/CalculatedContent/WeightWatcher https://github.com/CalculatedContent/WeightWatcher Please give it a try and let me know if it is useful to you

thisnormaluser 5y ago
Nature Communications is not Nature. But whatever.
- chaxor 5y ago
  I actually have a preference towards Nature Communications over Nature, as Nature Communications actually allows more pages for explanation and as such typically has more rigorous research. On the flip side, Nature often has bolder claims with less convincing evidence, which often reads similar to pop-sci articles.
- deleted 5y ago
  [deleted]
- deleted 5y ago
  [deleted]
- charleshmartin 5y ago
  That's fine. Here's a few of my online talks: UC Berkeley / ICSI: https://www.youtube.com/watch?v=6Zgul4oygMc https://www.youtube.com/watch?v=6Zgul4oygMc Stanford ICME: https://www.youtube.com/watch?v=PQUItQi-B-I https://www.youtube.com/watch?v=PQUItQi-B-I and a couple of Mikes's Institute for Pure & Applied Mathematics (IPAM) : https://www.youtube.com/watch?v=fmVuNRKsQa8 https://www.youtube.com/watch?v=fmVuNRKsQa8 Physics Informed Machine Learning: https://www.youtube.com/watch?v=eXhwLtjtUsI https://www.youtube.com/watch?v=eXhwLtjtUsI and our KDD workshop KDD: https://dl.acm.org/doi/abs/10.1145/3292500.3332294 https://dl.acm.org/doi/abs/10.1145/3292500.3332294 many more talks and papers available And my favorite, a podcast that has featured LeCun himself: https://blog.rebellionresearch.com/blog/theoretical-physicist-dr-charles-martin-on-deep-learning https://blog.rebellionresearch.com/blog/theoretical-physicis...

randcraw 5y ago

The article is intriguing, but is the objective of the mathematicians of more than theoretical value? It seems to me that proving DNNs to be equivalent to SVM kernels won't improve them appreciably -- they won't become more accurate or precise, not faster, nor more economical. More importantly, the achilles heel of DNNs won't be fixed -- their inability to explain their reasoning or annotate their inner workings to enable tuning or repair. Replacing black box linear algebra with black box math does not seem like much of a step forward.

charleshmartin 5y ago
agreed!
JustFinishedBSG 5y ago
> is the objective of the mathematicians of more than theoretical value? Why should it have any other value?

light_hue_1 5y ago

> By all accounts, deep neural networks like VGG have way too many parameters and should overfit. But they don’t. This isn't true at all. You can only make this statement relative to a dataset. And VGG is almost always trained on ImageNet. A standard VGG16 has 138 million parameters. ImageNet has 1 million images each is 244x244 (~50k). There are ~431 input data points per parameter in VGG16. VGG is not overparameterized for ImageNet. There are plenty of network/dataset pairs they could have given as an example, but this isn't it.

marcinzm 5y ago
VGG also has regularization (L2, dropout, maxpool) if I remember correctly so it's sort of odd to use that as an example of overfitting when it explicitly tunes against that. I'm guessing the epochs and learning rate were also tuned to lower overfitting. The same is true to every other popular network out there.
derbOac 5y ago
I have so many questions about this now. For example, how often are people not distinguishing between amount of data versus number of cases? Do we really have any sense of how "effective number of parameters" scales with network features? This article had me wondering if peoples' understanding of these things is backward or something.
- light_hue_1 5y ago
  > I have so many questions about this now. For example, how often are people not distinguishing between amount of data versus number of cases? To state your question more broadly. Data has structure that's why we can learn in the first place. Can we separately talk about the structure vs number of examples of the different structures in our data? In computer vision this would be sort of like talking about how much information is within an object class vs between object class. We don't know. And we don't have the mathematical tools to talk about this today. > Do we really have any sense of how "effective number of parameters" scales with network features? This article had me wondering if peoples' understanding of these things is backward or something. It's not backward, it's non-existent. We basically try things and when we have too many parameters we see that performance isn't increasing (or isn't increasing enough to be worth it, or with unsupervised methods and massive datasets we just max out the size of network we can train).

qPM9l3XJrF 5y ago

>By all accounts, deep neural networks like VGG have way too many parameters and should overfit. I keep seeing this claim pop up but I've never seen a citation to support it. My understanding is that e.g. statistical learning theory lets you prove theorems of the form "IF it has this number of parameters or fewer, THEN it won't overfit". First year logic is enough to know this theorem tells you nothing in the scenario where the model has lots of parameters. Can anyone provide a solid citation to support the claim that models with many parameters "should" overfit? BTW, another example of a model with lots of parameters which doesn't overfit is random forests / gradient boosting. I'm wondering if this "should overfit" claim is less of a global property of machine learning models in general, and more just a property of the particular models that this set of researchers have most familiarity with (SVMs).

JustFinishedBSG 5y ago
Yes usual results in statistical learning theory are upper bound on the generalization error, so if you have an "overparametrized" (number of parameters is actually a useless metric but that's another discussion altogether) then the best you can say is "well I expect the error to be at most A FREAKING LOT", which is not a very interesting result.

JustFinishedBSG 5y ago

I find the majority of the comments in this thread pretty concerning and quite honestly shocking. I didn't expect HN to be basically so obscurantist and to act so similarly to medieval alchemists (and that's not being fair to alchemists). So apparently theory is useless, all that matters is the "art" and empirical results ? Especially since quite a few of the comments here dismiss this work, and related, with a quick "well it's trivial/ well known?" and then procede to give their own personal explanation that show a clear misunderstanding of the topic.

quietbritishjim 5y ago
I don't have any skin in this game, but it's clear even to me that the comparison with alchemists isn't valid: neural nets, and CNNs in particular have achieved actual results, whereas alchemists did not.

sgt101 5y ago

By the time I finished my Ph.D. in 1997 I was stone cold certain that SVM's were equivalent to NN's, and that 3 layers could do anything. I saw Y.Lecuns work on handwriting and thought it an oddity. I gave up on machine learning as I thought it was all done and concentrated on applications. In 2011 I was staggered to realise what a load of bollocks I had been bongoogled into believing in, and annoyed. I am never going to fall for the "if we assume x, and y and only talk about this bit of the system then it's all the same" line again. It may be mathematically nice to prove things with simplifying assumptions, but the ICML community basically walked up a blind ally because of this desire for rigor and elegance in preference to reality. ML is not mathematics, because it's tied to data, and data is a product of nature. The study of nature is Science. This is the difference between String Theory and Physics, and it's the difference between Computer Science and ML as well.

jhgb 5y ago
Isn't it still the case mathematically that 3 layers can do theoretically anything? Which of course may be a different claim that they can do that practically.
- amelius 5y ago
  > Isn't it still the case mathematically that 3 layers can do theoretically anything? What do you (and parent) mean by "do"? Inference, or training, or both?
  - sgt101 5y ago
    The point is that a three layer network can approximate any function, and back propagation can be used to train it to do so. You could train a 3 layer net to do imagenet, I think it was Minsky that showed the first bit of that in "Perceptrons", and Hinton showed it in his backprop paper with Rumelheart (?) but I can't be arsed to look it up, someone will pop up and show that it was actually Leonardo Da Vinci or that German fella who has apparently invented everything ever and never tires of telling everyone about it no matter how hard we all ignore him. However, clever people (Hinton, Lecun, others v.smart and determined and all should get a lot of credit) invented some tricks to train networks faster (and carefully showed that they really worked, despite what people said about them, especially people at ICML), and computers got many faster, and it became obvious (which it was to anyone who both could do the sums and had the imagination to think about it - which I didn't, at all, sad for me) that broad networks are much much slower to train than deep networks for an equivalent number of neurons (if you use tricks like regularization and dropout) and this makes large deep networks feasible. I did play with deep networks in like 1994, because I wrote a C program to let me write neural networks - and I could do anything I liked with it on my Sun workstation and my supervisors had no idea about anything. However, I set this "deep" network off training to see what would happen and went to the pub. When I came back I interrupted it and looked at the model file to figure out how much the parameters had changed since initialization (I didn't have any of the good small value initialization stuff, I think I set them to 0) and they had changed so little that I did some sums and decided that the model would not be finished training before I finished grad school if I finished at all. So, I went off and built various types of booster on an MPI network that most of my colleagues didn't realize was running on their nice Sun workstations instead. Talking to other people in pubs at conferences has since convinced me that most of the grad students working on ML in the 90's did the same thing.

YeGoblynQueenne 5y ago

The article leaves "generalisation" without further explanation and as usual, there's a lot of confusion about what, exactly, is meant by the claim that overparameterised deep neural networks "generalize astoundingly well to new data". So, what is meant is that such networks (let's call them complex networks for short) generalise well to test data. But what exactly is "test data"? Here's how it works. A team has some data, let's call it D. The team partitions the data to a training and test set, T₁ and T₂. The team trains their system on T₁ and tests it on T₂. To choose the model to test on T₂, the team may perform cross-validation which further partitions T₁ to k validation partitions, T₁₁, ..., T₁ₖ. In the most common cross-validation scheme, k-fold cross-validation, the team trains on k-1 validation partitions of T₁ and tests on one of the k validation partitions, trying all combinations of k-1 training and 1 testing, partitions _of T₁_ (because we're still in validation, which can be very confusing). At the end of this process the team have k models each of which is trained on k-1 different partitions, and tested on one of k different partitions, of the training set T₁. The team chooses the model with the best accuracy (or whatever the metric they're measuring) and then they test this model on T₂, the testing partition. Finally, the team report the accuracy (etc) of their trained model on the testing partition, T₂, and basically claim (though generally the claim is implicit) that the accuracy of their best model on T₂ is a more or less accurate estimate of the accuracy of the model on unseen data, in the real world, i.e. data not included in D, either as T₁ or T₂. Such truly unseen data (it was not available _to the team_ at training time) is sometimes referred to as "out of distribution" data and let's call it that for simplicity [1]. So, with this background at hand, we now understand that when the article above (and neural net researchers) say that complex networks "generalise well", they mean "on test data". Which is mildly surprising given that various assumptions make it less surprising that they'd do well on training data etc etc. There are two key observations to make here. One is that complex networks generalise well on test data _when the researchers have access to the test data_. When researchers have access to test data, the training regime I outline above includes a further step: the team looks at the accuracy of their system on test data, find it to be abysmal, and, crestfallen, abandon their expensive research and start completely from scratch, because obviously they wouldn't just try to tweak their system to perform better on the test data! That would be essentially peeking at the test data and guiding their system to perform well on it (e.g. by tuning hyperparameters or by better "random" initialisation)! I'm kidding. Of course a team who finds their system performs badly on test data will tweak their system to perform better on the test data. They'll peek. And peek again. Not only they'll peek, they'll conduct an automated hyperparameter-tuning search (a "grid search") _on the test data_! That is what's known in science, I believe, as a "fishing expedition". Two, in the typical training regime, the training partition T₁ is 80% of D, the entire dataset, and T₂ is 20% of D. And when T₁ is partitioned for cross-validation, the validation-training partition (the k-1 folds trained on) is also usually 4 times larger than the validation-testing partition (the single fold tested on). Why? Because complex networks need Big Data to train on. But, what happens when you train on 80% of your data and test on 20% of it? What happens is that your estimates of accuracy are not very good estimates, because, even if your data is Really Big, 20% of it is likely to miss a big chunk of the variation found in the 80% of the data, and so you're basically only estimating your system's accuracy on a small sub-set of the features that it needs to represent to be said to "generalise well". For this reason, complex networks, like all Big Data approaches, are pretty much crap at generalising to "O.O.D." data, or, more to the point, estimates of their accuracy on OOD data are just pretty bad. In practice, deep neural net systems that have amazing performance "in the lab", can be expected to lose 20-40% of their performance "in the real world" [2]. Bottom line, there's no big mystery why complex networks "generalise" so well and there is no need to seek an explanation for this in kernel machines. Complex networks generalise so well because the kind of generalisation they're so good at is the kind of generalisation achieved by humans tweaking the network until it overfits _to the test data_. And that's the worst kind of overfitting. __________ [1] Under PAC-Learning assumptions we can expect the performance of any machine learning system on data that is truly "out of distribution", in the sense that it is drawn from a distribution radically different than the distribution of the training data, to be really bad, because we assume distributional consistency between training and "true" data. But that's a bit of a quibble, and "OOD" has established itself as a half-understood jargon term so I let this rest. [2] I had a reference for that somewhere... can't find it. You'll have to take my word for it. Would I lie to you?