13 ms·

Why deep learning works even though it shouldn’t

356 points by r4um 6y ago

yters 6y ago
How do we define 'works'?
- quotemstr 6y ago
  Are you not familiar with how awful machine translation was ten years ago and how eerily good it is today?
  - pessimizer 6y ago
    I'm definitely not familiar with that. From my perspective it was mediocre but workable 10 years ago, and it's marginally better now if the languages are closely related.
    - chillacy 6y ago
      There exist more objective measurements of translation quality which if you take to be relevant, show that machine translation quality has gotten better between most languages. https://en.wikipedia.org/wiki/Evaluation_of_machine_translation https://en.wikipedia.org/wiki/Evaluation_of_machine_translat... Here's a blogpost by google showing scores creeping up since 2009 https://ai.googleblog.com/2020/06/recent-advances-in-google-translate.html https://ai.googleblog.com/2020/06/recent-advances-in-google-... Also I agree that I don't recall it being awful like the parent poster suggests but maybe only because I reserve 'awful' for the type of rule based machine translation used before statistical approaches came onto the scene.
  - anfilt 6y ago
    It still seems pretty aweful today for a lot stuff i try to translate.
  - anfilt 6y ago
    It still seems pretty aweful today for a lot stuff i try to translate with it
  - morei 6y ago
    I think voice recognition would be a better example. It went from 'toy' to 'everyday use' because the word error rate dropped an order of magnitude.
    - yters 6y ago
      Is that due to ML or all the massive crowdsourcing and more keyword/search driven approach? e.g. has dictation apps like dragon improved substantially?
      - morei 6y ago
        Some of it is due to bigger data, but the majority is definitely ML. For constant data, the error rate dropped dramatically due to much improved algorithms. 15 years ago, voice recognition was all hidden markov model based. The data sets were limited, in part due to the cost of collection, but mostly because larger datasets didn't significantly improve accuracy. As algorithms improved, larger datasets became more important as more data did actually improve accuracy.
      - jaimex2 6y ago
        Both funnily enough. ML has been around for years, the main reason it's getting better is that it's easier than ever to collect massive amounts of data to train good models.
  - yters 6y ago
    i would say that's more a function of corpus than ML plus is it really that much better? i still get weirdness out of google translate
    - PeterisP 6y ago
      Nope, we have loads of experimental evidence (and I mean, it's something one can verify at home for smaller datasets, it does not take that much compute) that neural MT gets significantly better results than what we could (and still can) achieve with "pre-neural" SMT methods on the exact same corpus. A general benchmark for comparing the effectiveness of different MT approaches are the WMT conference series (e.g. http://www.statmt.org/wmt20 http://www.statmt.org/wmt20) shared tasks where the systems are trained on the same corpus. They're still not perfect, and sure, you get weirdness, but it has become significantly better according to all metrics including human comparisons of different translation aspects, which are expensive/rare to do but have been done quite a few times; there's a clear consensus that deep learning "works" for ML. There are certain niches where other methods may still be better (IIRC languages with very little data, and translation of specific 'controlled language' domains), but for mainstream MT I think that nowadays nobody would decide to build a non-neural system.
  - TillE 6y ago
    Japanese and Chinese translation to English is still absolute garbage outside of some obviously targeted sites (Wikipedia, I believe). Translating a German news site is still riddled with key errors because it doesn't understand how words and phrases get used in different contexts. It's improved a bit, but it's still miles and miles away from any half-decent human translation.
quicklime 6y ago
> High dimensional spaces are unlikely to have local optima, and probably don’t have any optima at all. Can someone who knows more about DL than I do help me understand this a little better? The article uses the analogy of walls: > Just recall what is necessary for a set of parameters to be at a optimum. All the gradients need to be zero, and the hessian needs to be positive semidefinite. In other words, you need to be surrounded by walls. In 4 dimensions, you can walk through walls. GPT3 has 175 billion parameters. In 175 billion dimensions, walls are so far beneath your notice that if you observe them at all it is like God looking down upon individual protons. I'm struggling to understand what this really means in 4+ dimensions. But when I try to envision it going from 1 or 2 to 3 dimensions, it doesn't seem obvious at all that a 3D space should have fewer local optima than a 2D space. In fact, having a "universal function" like a deep network seems like it should have more local optima. What am I missing?
- moultano 6y ago
  Think of the model as a point in high d space that you're trying to trap inside a cage. (Corresponding to it being at a local minima and surrounded by higher points.) So you're trying to build a cube in n dimensions, which has 2n sides. Now imagine that each of those sides will randomly be there or not, probabilistically, with probability p. To actually trap a model like gpt3, p^175,000,000,000 has to be high enough that you observe a case during training where you roll a 1.
- zachf 6y ago
  The “physicist” explanation that I heard (meaning non-rigorous but good for building intuition) is that at every point where the derivative vanishes, for suitably random functions, every direction you move in will either be a direction where you increase or decrease at about 50-50 odds. In D dimensions there are 2D independent directions to rise or fall in (e.g. in D=2 dimensions, there’s north south east west), so there’s about 1/2^(2D) odds that any given critical point is actually a minimum if those probabilities are independent. That gets small really fast at large D. Obviously this is not rigorous, though.
  - maxlamb 6y ago
unimpossible 6y ago
A lot of people talk about minima because thats the language we have for analyzing optimization techniques. Deep Learning is still new enough that there is lots of low hanging fruit to explore including empirical approaches and applying existing theoretical tools to try and explain DNNs. The community is slowly moving towards developing new tools specifically for deep learning properly analyze these networks and prove stuff (bounds, convergence etc) about them.
jostmey 6y ago
The author argues that deep-learning has abandoned statistics. I could not disagree more! Too much of the field was concerned with detailed proofs and mathematical formalism that were somewhat disconnected from probability theory. Modern machine learning (or AI or whatever) still has strong roots in probability and statistics. Loss functions are still based on concepts such as the log-likelihood function. Formal proofs and mathematics are essential, but can become a distraction from the end goal. It is like playing Chess by going after your opponents pawns instead of their king. I would say modern machine learning has become tantamount to experimental physics and this article is written from the perspective of a string theory theorist.
- moultano 6y ago
  That isn't what I'm arguing at all.
  - candiodari 6y ago
    I think this is a way to really piss off statisticians: Stop talking about minima. Stop talking about how your optimization algorithm behaves around a minimum. Nobody ever trains their model remotely close to convergence. ADAM doesn’t even provably converge. All real models diverge! You are nowhere close to a minimum! Stop talking about minima already goddamnit! Why even think about minima?! Minima are a myth! Everybody proves their results for minima of a convex function. Actually arguing for divergent models. I mean, I love it, just because it's so very very cruel.
fizixer 6y ago
Woah! talk about sh-tting on deep learning and knowing jack about double descent, deep double descent. flag-worthy sh-tpost.
RandyRanderson 6y ago
Certainly a much shorter way to say this: if you have enough lines you can approximate any curve within a margin. This is what large neural networks are doing. Another way to look at it: most neural nets are just a bunch of polynomials stitched together. You can see this from the popularity of the relu activation function. when the relu is negative, that poly is always zero in that area. When positive it's some poly multiplied by a const - another poly. For nets that use other activation fns, they try to be linear in the area of most active input. So again they approximate a const * a ploy.
- dougabug 6y ago
  ReLU networks are piecewise linear.
  - RandyRanderson 6y ago
    Have a look at some published nn architectures - you'll typically see a few nonlinear units/transforms and a LOT of ReLU ones.
- blackbear_ 6y ago
  You are missing the point. The surprising thing about deep learning is that it can generalize to unseen data so well. Polynomial regression cannot.
  - mamon 6y ago
    If there is a pattern in data, such as it fitting on a curve, and you approximate that curve, then that should generalize to unseen data. What's surprising about that? A single polynominal regression wouldn't be able to do it, because some curves cannot be expressed as a polynominal, but superposition of multiple polynominals is apparently good enough.
    - moultano 6y ago
      If that worked, people would use it, but it doesn't, so they don't.
acadien 6y ago
Hey @moultano in response to your argument about walls and Nets not being in a minima, its my understanding nets always live on high dimensional saddle points and that's commonly referred to in literature. Even when you're optimizing you're just moving towards ever lower cost saddles that are closer to the optimum but almost never a local optimum (for the reasons spelled out in your post).
- moultano 6y ago
  Thank you. Several people have pointed that out, and I'm probably not reading the right papers. Is it common when people introduce a new flavor of adaptive SGD to address how it handles saddles specifically? It is probably just a a matter of what manages to bubble up to me rather than what work is actually getting done, but I felt like the non-convergence of ADAM got talked about a lot, but haven't seen people talking as much about how optimizers behave differently on the landscapes we actually observe.
  - acadien 6y ago
    Saddles are a way of conceptualizing high dimensional optimization problems. If you have a 3 dimensional surface you can imagine a saddle as an isocurve that follows a minima in at least one dimension. Another way to conceptualize these is to think of being at the minima of a parabola in 2 dimensions, but then seeing you're not in a minima in a 3rd dimension. Any time you're in a minima in at least 1 dimension, you're on a saddle. You can extend this concept to a neural net which lives in millions of dimensions, undergoing SGD. When beginning an optimization run SGD moves in some direction to minimize the a bundled cost, inevitably stumbling into minima in (usually) many dimensions. Subsequent iterations will shift some dimensions out of minima and other dimensions into minima, the net is always living on a saddle during this process. There are many papers that discuss the process in these terms and others that implicitly use it. I wouldn't say its a "hot area of research" but more of a tool for thinking about these processes and sometimes gaining some insight in to why things get stuck during training.
- muppet_frog 6y ago
  This paper makes the points that it's the saddles and not local minima that are the problem: https://arxiv.org/abs/1406.2572 https://arxiv.org/abs/1406.2572 It was the basis for adding 'momentum' to optimizers - so that you could skate across the saddles.
21eleven 6y ago
I liked that part at the beginning where the author made clear they were going to discuss intuitions that, while they aren't proven, would be useful to make explicit for a more general audience. Good candor.
- FriendlyNormie 6y ago
  Good job repeating the second paragraph with slightly different words. What would we do without your brilliant contribution to the discussion?
blackbear_ 6y ago
There is another reason why training deep neural networks is not as difficult as it sounds: the landscape of the loss function seems to be made of broad "U"-shaped valleys that gently descend towards a small loss region. At initialization, the network is likely close to such valley, and once it gets there the rest of training is just a leisurely stroll. Formally, people have studied the spectrum of the Hessian and found that most of its eigenvectors are quite small with only a few, much larger ones. It all started with [1], with several recent extensions. [1] https://arxiv.org/pdf/1611.07476.pdf https://arxiv.org/pdf/1611.07476.pdf
- aborsy 6y ago
  This is just bunch of simulation, not a derivation.
  - blackbear_ 6y ago
    True, but it has been confirmed experimentally several times. Unfortunately theoretical understanding in deep learning is quite lagging behind practical progress. This paper [1] seems to be getting closer to what you are looking for. [1] https://arxiv.org/abs/1910.02875 https://arxiv.org/abs/1910.02875
- hammock 6y ago
  Explain in layman's terms?
  - macksd 6y ago
    Picture linear regression. If you have a bunch of data points you want to fit a line to, for any given line you can add up the vertical distance between all your data points and the line and come up with a measure of how inaccurate the line is. This is called your "loss". If you keep trying different values for the slope and intercept, you would find that this function is shaped like a big bowl, or valley. Regression is the process of repeatedly taking a step downhill until you can't go anywhere but up, and that must be the optimal line. Neural networks train in a similar way. You have a "loss" function that adds up how wrong your predictions are compared to the training data. You try different values for the weights in your neural network to see which ones send you downhill the fastest, step them in that direction, and repeat. Since the loss function in this case is more complex, it's not a single valley, but potentially many valleys. You can end up at a decent local minimum.
Schiphol 6y ago
I loved this post, thanks for writing it. I get the argument why one shouldn't expect local minima in very high dimensions. But then, what's wrong with the informal argument that there has to be a minimum because, well, the expected loss cannot be negative?
- moultano 6y ago
  With squared loss where it's easy for the loss to be zero, then yes, it will have lots of global minima, all at a loss of zero. For losses that asympotote, like logloss, they may have no minima.
  - Schiphol 6y ago
    Thanks. I guess my worry was that, once you are doing extremely well and your loss is very low, gradients are no longer independent, and will tend to go mostly up. Is this wrong?
    - joshribakoff 6y ago
      I’m an ML newb but I think this would be true only of a converged model. Your model could always technically diverge in another epoch if learning rate is high enough and you process a batch of extreme outliers Even then, you may have still converged on a local optimum which was the take away I got from the article
woopwoop 6y ago
It seems to me all of these arguments apply just as well when the "deep" network has only one hidden layer.
- moultano 6y ago
  That's true. I left out saying that deeper networks represent a wider variety of functions more easily, because that seems generally intuitive to everyone. But the arguments about how easy it is to optimize them should apply equally well to wide and shallow networks as to very deep ones.
  - woopwoop 6y ago
    Thanks for the response. Hmm, it's still pretty mysterious to me. Why should a deep network with the same number of parameters as a wide network represent a wider variety of functions? In some sense they represent the "same" number of functions, in the sense that the manifold of functions given by two network architectures with the same number of parameters have have the same dimension, even if one is wide while the other is deep.
    - bananaface 6y ago
      I think a deeper network has less degrees of freedom in which to move, or rather, in which to move usefully, because parameters are more interdependent. That means in order to generate a useful function, it has to learn more abstract features than a shallow and wide network. This is because any adjustment to irrelevant features that are unique to a small number of examples has a larger negative impact on the rest of the data than it would in a shallower net (except in the latter layers). Over time, abstract changes are stochastically rewarded and specific changes are penalised, at least for the earlier layers, and the latter layers then have to work with this more abstract information so they simply can't overfit that much. Would be interested in OP's take on this though.
brundolf 6y ago
Setting aside the primary subject, this is an excellent observation: > What I find however is that there are a base of unspoken intuitions that underlie expert understanding of a field, that are never directly stated in the literature, because they can’t be easily proved with the rigor that the literature demands. And as a result, the insights exist only in conversation and subtext, which make them inaccessible to the casual reader.
- wenc 6y ago
  This is very true. This is why conferences are important. A lot of knowledge is sociological in nature. Failures and tricks of the trade are discussed at the bar after 5. I've experienced this first hand. Academic publications and lectures, in their (rightful) pursuit of rigor, aren't usually the right space for conversations around hunches and experiences. Reputations are at stake, and most people are doing impression management. When experienced readers read a journal publication, they read it with the implicit understanding that it is a highly curated view of the messiness behind the scenes.
  - brundolf 6y ago
    It seems toxic to me that there's no accepted public venue for that stuff, though. The fact that people are too afraid to relay certain useful information until they're tipsy; the emphasis on "doing impression management". Maybe it should remain separate from the rigorous stuff, but where's the "Op-Ed section" of academic publishing?
    - wenc 6y ago
      I feel the word "toxic" is maybe a bit too pejorative for what it is? Another framing is that it is "guild knowledge". The incentives in academia are complex and it affects how open some folks are or can afford to be with such knowledge. That said, there are certain open avenues for making such knowledge public. MathOverflow is one. Some academics document their guild knowledge in "technical papers" which they put up on their website. The ML community (which I'm not part of, but that I'm able to observe as an outsider) seems to be particularly open when it comes to publishing blog posts -- sometimes to gain reputational points? In some journals, arguments over publications are carried out in the Letters to the Editor section. Sometimes this leads to public feuds however, and some academic communities are small enough that if you make too many enemies your publications may be visited upon with disfavor when it comes time for peer review. It's not worth getting into public tiffs unless there's a principle at stake.
cs702 6y ago
TL;DR: For high-dimensional models (say, with millions to billions of parameters), there's always a good set parameters nearby, and when we start descending towards it, we are highly unlikely to get stuck, because almost always there's at least one path down along at least one among of all those dimensions -- i.e., there are no local optima. Once we've stumbled upon a good set of parameters, as measured by validation, we can stop. These intuitions are consistent with my experience... but I think there's more to deep learning. For instance, these intuitions fail to explain "weird" phenomena, such as "double descent" and "interpolation thresholds": * https://openai.com/blog/deep-double-descent/ https://openai.com/blog/deep-double-descent/ * https://arxiv.org/abs/1809.09349 https://arxiv.org/abs/1809.09349 * https://arxiv.org/abs/1812.11118 https://arxiv.org/abs/1812.11118 * See also: http://www.stat.cmu.edu/~ryantibs/papers/lsinter.pdf http://www.stat.cmu.edu/~ryantibs/papers/lsinter.pdf We still don't fully understand why stochastic gradient descent works so well in so many domains.
YeGoblynQueenne 6y ago
I find the article style unreadable. Could someone please say whether the author explains why deep learning shouldn't work? There is a bit at the start about how people in statistical learning throw their hands up at deep learning etc, but none of that makes sense to me. Neural nets are an idea as old as AI - even older, in fact. The need for deeper networks was well understood by the 1980's. There are well known results about feedforward neural nets with arbitrary hideen units being universal function approximators. Why shouldn't deep learning work?
- moultano 6y ago
  People coming from the perspectives of general optimization believed that they were impossible to train, due to being very non-convex. People coming from the perspective of classical statistics believed that they couldn't generalize due to needing large numbers of parameters. Both of those turned out to be very wrong, and this post is trying to explain why.
  - YeGoblynQueenne 6y ago
    Thank you for the summarisation. Does the article give any examples of such arguments, or is it something stated as a commonly known fact? Coming from an artificial intelligence background I am not aware of such opinions. I know that deep neural nets were considered difficult to train until the re-discovery of backpropagation, but not because of anything to do with the shape of the error function. However, as usual there is confusion about what "generalisation" means. For example, I was in a summer school at Oxford a couple of years ago and one of the lectures made a similar point, about the surprising generalisation ability of deep neural nets. I approached the lecturer after the lecture and asked what they meant because the way I knew it, neural nets can't generalise, and they explained that they meant that they generalise surprisingly well on the test set but not on unseen data, or out-of-distribution data, i.e. not on any data that was not available to the researcher during training (as training, validation or test set). In other words, neural nets are great at "generalisation" in the sense of interpolation, but are almost completely incapable of "generalisation" in the form of extrapolation. I like to quote Francois Chollet of Keras on this: This stands in sharp contrast with what deep nets do, which I would call "local generalization": the mapping from inputs to outputs performed by deep nets quickly stops making sense if new inputs differ even slightly from what they saw at training time. Consider, for instance, the problem of learning the appropriate launch parameters to get a rocket to land on the moon. If you were to use a deep net for this task, whether training using supervised learning or reinforcement learning, you would need to feed it with thousands or even millions of launch trials, i.e. you would need to expose it to a dense sampling of the input space, in order to learn a reliable mapping from input space to output space. By contrast, humans can use their power of abstraction to come up with physical models—rocket science—and derive an exact solution that will get the rocket on the moon in just one or few trials. Similarly, if you developed a deep net controlling a human body, and wanted it to learn to safely navigate a city without getting hit by cars, the net would have to die many thousands of times in various situations until it could infer that cars and dangerous, and develop appropriate avoidance behaviors. Dropped into a new city, the net would have to relearn most of what it knows. On the other hand, humans are able to learn safe behaviors without having to die even once—again, thanks to their power of abstract modeling of hypothetical situations. https://blog.keras.io/the-limitations-of-deep-learning.html https://blog.keras.io/the-limitations-of-deep-learning.html In short, if the point of the article is that neural networks "work" because they generalise in the sense of extrapolation, then that's not right.
maxwells-daemon 6y ago
I don't think we can say for sure that early stopping is the main reason deep networks generalize. Double descent [1] shows that models continue to improve even once they've "interpolated" the training data (fit every point perfectly), and critical periods [2] suggest that the early part of training is responsible for most of the generalization performance even though much of the numerical improvement happens later. Overall it looks like gradient descent is a strong regularizer -- we know it tends to prefer small and low-variance weights, for example. So part of deep generalization has to do with how SGD is able to pick "good" features early, and then optimization pushes the unimportant weights to zero later (hence lottery tickets). [1] https://openai.com/blog/deep-double-descent/ https://openai.com/blog/deep-double-descent/ and other papers. [2] https://arxiv.org/abs/1711.08856 https://arxiv.org/abs/1711.08856 and others.
- owenshen24 6y ago
  To add some more context, here's a rather readable summary: https://www.greaterwrong.com/posts/FRv7ryoqtvSuqBxuT/understanding-deep-double-descent https://www.greaterwrong.com/posts/FRv7ryoqtvSuqBxuT/underst...
- machinelearning 6y ago
  I think you misunderstood the point of deep double descent. The x-axis is not number of training epochs, it is model capacity. I think you'd be interested in https://arxiv.org/abs/1611.03530 https://arxiv.org/abs/1611.03530. It discusses how SGD is an implicit regularizer. We also actually want high variance weights for symmetry breaking.
Animats 6y ago
Now that's fascinating. I'd thought of machine learning as a form of optimization. Things like support vector machines really were hill climbing for some kind of local optimum point. But at a billion dimensions, you're doing something else entirely. I once went through Andrew Ng's old machine learning course on video, and he was definitely doing optimization. The last time I actually had to do numerical optimization using gradients, I was trying to solve nonlinear differential equations for a physics engine in about a 20-dimensional space of joint angles that was very "stiff". That is, some dimensions might be many orders of magnitude steeper than another. It's like walking on a narrow zigzagging mountain ridge without falling off. So deep learning is not at all like either of those. Hm.
mrfusion 6y ago
This quote: there are a base of unspoken intuitions that underlie expert understanding of a field, that are never directly stated in the literature, because they can’t be easily proved with the rigor that the literature demands. And as a result, the insights exist only in conversation and subtext, which make them inaccessible to the casual reader. I’d love to hear these intuitions from every field. Anyone got some?
- Fordec 6y ago
  All sensor data, the closer you get to the analog side of things, is bullshit. It's just about smoothing over the bullshit enough to make the tolerances workable for real world applications. We call this bullshit smoothing "calibration". If you're doing work on sensor data and don't have every calibration parameter, whether from configuration or magic factory numbers and statistical tolerances, someone, somewhere is pulling the wool over the eyes of the software guy downstream that works with the final data. Ever looked at weather data from two separate apps and have values vary by multiple degrees? Two different pipeline just sprinkled on their own versions of interpretations on top of raw sewage data.
  - brandmeyer 6y ago
    > Ever looked at weather data from two separate apps and have values vary by multiple degrees? Actually this is because surface temperatures really do have this kind of variation.
ummonk 6y ago
Great article, though I never understood why people would think deep learning shouldn't work.
deleted 6y ago
[deleted]
KKKKkkkk1 6y ago
But it doesn't. Researchers have been saying for several years now that computer vision is more accurate than human vision, and face recognition was one of the first problems "solved." And yet when the pandemic hit, Apple had to scramble to adjust its unlock mechanism in iOS 13.5 because Face ID cannot recognize people wearing masks [1]. Humans have no trouble identifying people wearing masks. We are now almost a year into the pandemic, iOS 14 has been released, Face ID has not been fixed, and now we hear that Apple is bringing Touch ID back [2]. So sure, you've developed a methodology that can overfit nicely not only the train data but even the test data. But it still fails miserably when you apply your model in the field. [1] https://www.theverge.com/2020/5/20/21265019/apple-ios-13-5-out-now-unlock-iphone-face-mask-id-exposure-notification-covid-19 https://www.theverge.com/2020/5/20/21265019/apple-ios-13-5-o... [2] https://appleinsider.com/articles/20/10/16/under-display-touch-id-on-an-iphone-is-still-coming-leaker-claims https://appleinsider.com/articles/20/10/16/under-display-tou...
- why_only_15 6y ago
  I don't think this is quite fair. Of course a machine learning model doesn't work on things it wasn't trained for. Computer vision is not more accurate than human vision in every possible field. And are you really sure humans have no trouble identifying people wearing masks? Lots of people look pretty similar, especially if all you can see is their face.
- rajansaini 6y ago
  Intuitively, it seems like voice signatures, body language (like walking habits), and height/weight would play a larger role in helping humans identify a masked person than exposed facial features. Because the network behind Apple's facial recognition software cannot have access to this kind of data (well, maybe voice, but that doesn't seem secure), I'm not sure this is a fair comparison. Would love to be refuted, however.
  - teorema 6y ago
    I don't know that I'm looking to refute you per se but... A better example than face masks maybe is the recent controversy over Twitter's AI and Obama images (https://www.theverge.com/2020/9/20/21447998/twitter-photo-preview-white-black-faces https://www.theverge.com/2020/9/20/21447998/twitter-photo-pr...). A lot was made of racial issues, which is fine, but the broader issue is why subtle changes in photos, like cropping, should confuse things so completely. The target piece (the focus of this HN thread) sums itself up this way: "There’s a good set of params somewhere nearby. When we start walking to it, we can’t ever get stuck along the way, because there are no local optima. Once we’ve stumbled upon a good set of parameters, we’ll know it and we can just stop." I think there's some useful insights there, but this is in many ways the definition of local optima. What I might argue is that because there's so many locations in high-dimensional space that will satisfy some classification goal, it's "easy" to find one that works with regard to some population that defines the model development space (training + test). However, that model development space/population is itself implicitly defined by a certain set of constraints -- it's a subpopulation of some broader population. What you want to generalize to to define overfitting is broader. You can still not overfit to your model development space, but be overfitting with regard to some broader set of possible inputs. Whether or not the constraints of the model development population/space are important and reasonable considerations -- e.g., in your argument, not having access to things like body language etc -- is maybe a little variable. In some cases the implicit defining characteristics of the model development population are meaningful, but in other cases they're hidden. In Twitter's case, you end up finding out later that there's weird things that probably defined the space of their images that they didn't intend. It's only in the adversarial case that you learn about this. In classical statistics, you talk about generalization and overfitting, but there's an implicit population you're sampling from that defines both of those things. That is, you have a training/fitting/initial sample, and you ask yourself how well your model would perform on a test/validation sample. But implicit in that is some assumption about what it means to be a random sample from the same population. I think lots of times with DL, the cross-validation/test sample is also implicitly defined as coming from some population. But the population isn't some model, it's some source. Some image dataset, something like that. There will be things about that source that are "of interest", but other things that are idiosyncratic about it, and unrepresentative of the "real" population of interest. In this way, I'm not sure that held-out samples from some source are really the right way to think of generalization and overfitting -- I think the adversarial setting is the generalization setting. https://www.sciencemag.org/news/2020/05/eye-catching-advances-some-ai-fields-are-not-real https://www.sciencemag.org/news/2020/05/eye-catching-advance... Along the way from the classical to the DL regime I think there have been some overlooked issues about what it means to generalize, what you're really sampling from, and what your "population" actually is. It parallels tensions about theory versus experimentation because having a population in the classical case that you're sampling from requires a certain data-generating theory, which is often lacking in DL. The closest thing in the classical regime to DL generalization theory is maybe a sort of ultra-high-dimensional bootstrapping with random effects: showing that your bootstrap samples are close to your observed sample isn't the same thing as showing they're close to the population, or to other samples drawn from that population, especially in the presence of random effects.
hexo 6y ago
Blank page? :(
foxes 6y ago
There is a trick in physics I am reminded about. In infinite dimensions there is no way to have a Gaussian measure on just an infinite dimensional Hilbert space. It needs to be embedded inside a bigger infinite dimensional space and then you can have some relative measure. So you do not look at just a Gaussian integral, you look at a quotient of Gaussian integrals. Perhaps there is a similar idea. Perhaps there is some sort of renormalisation that would make neural networks work better. Even if your neural network is infinite dimensional it still makes sense to talk about some surface relatively.
ZeljkoS 6y ago
It looks like the article was deleted ("Oops! That page can’t be found"). Here is the Google Cache: https://webcache.googleusercontent.com/search?q=cache:5HMZ_ZO0mwkJ:https://moultano.wordpress.com/2020/10/18/why-deep-learning-works-even-though-it-shouldnt/+&cd=1&hl=en&ct=clnk&gl=hr https://webcache.googleusercontent.com/search?q=cache:5HMZ_Z...
turingbook 6y ago
The article was gone?
mikhailfranco 6y ago
Another interesting explanation for deep learning's success in the physical world: Why does deep and cheap learning work so well? Max Tegmark et al., https://arxiv.org/abs/1608.08225 https://arxiv.org/abs/1608.08225

356 points by r4um 6y ago

yters 6y ago

How do we define 'works'?

quotemstr 6y ago
Are you not familiar with how awful machine translation was ten years ago and how eerily good it is today?
- pessimizer 6y ago
  I'm definitely not familiar with that. From my perspective it was mediocre but workable 10 years ago, and it's marginally better now if the languages are closely related.
  - chillacy 6y ago
    There exist more objective measurements of translation quality which if you take to be relevant, show that machine translation quality has gotten better between most languages. https://en.wikipedia.org/wiki/Evaluation_of_machine_translation https://en.wikipedia.org/wiki/Evaluation_of_machine_translat... Here's a blogpost by google showing scores creeping up since 2009 https://ai.googleblog.com/2020/06/recent-advances-in-google-translate.html https://ai.googleblog.com/2020/06/recent-advances-in-google-... Also I agree that I don't recall it being awful like the parent poster suggests but maybe only because I reserve 'awful' for the type of rule based machine translation used before statistical approaches came onto the scene.
- anfilt 6y ago
  It still seems pretty aweful today for a lot stuff i try to translate.
- anfilt 6y ago
  It still seems pretty aweful today for a lot stuff i try to translate with it
- morei 6y ago
  I think voice recognition would be a better example. It went from 'toy' to 'everyday use' because the word error rate dropped an order of magnitude.
  - yters 6y ago
    Is that due to ML or all the massive crowdsourcing and more keyword/search driven approach? e.g. has dictation apps like dragon improved substantially?
    - morei 6y ago
      Some of it is due to bigger data, but the majority is definitely ML. For constant data, the error rate dropped dramatically due to much improved algorithms. 15 years ago, voice recognition was all hidden markov model based. The data sets were limited, in part due to the cost of collection, but mostly because larger datasets didn't significantly improve accuracy. As algorithms improved, larger datasets became more important as more data did actually improve accuracy.
    - jaimex2 6y ago
      Both funnily enough. ML has been around for years, the main reason it's getting better is that it's easier than ever to collect massive amounts of data to train good models.
- yters 6y ago
  i would say that's more a function of corpus than ML plus is it really that much better? i still get weirdness out of google translate
  - PeterisP 6y ago
    Nope, we have loads of experimental evidence (and I mean, it's something one can verify at home for smaller datasets, it does not take that much compute) that neural MT gets significantly better results than what we could (and still can) achieve with "pre-neural" SMT methods on the exact same corpus. A general benchmark for comparing the effectiveness of different MT approaches are the WMT conference series (e.g. http://www.statmt.org/wmt20 http://www.statmt.org/wmt20) shared tasks where the systems are trained on the same corpus. They're still not perfect, and sure, you get weirdness, but it has become significantly better according to all metrics including human comparisons of different translation aspects, which are expensive/rare to do but have been done quite a few times; there's a clear consensus that deep learning "works" for ML. There are certain niches where other methods may still be better (IIRC languages with very little data, and translation of specific 'controlled language' domains), but for mainstream MT I think that nowadays nobody would decide to build a non-neural system.
- TillE 6y ago
  Japanese and Chinese translation to English is still absolute garbage outside of some obviously targeted sites (Wikipedia, I believe). Translating a German news site is still riddled with key errors because it doesn't understand how words and phrases get used in different contexts. It's improved a bit, but it's still miles and miles away from any half-decent human translation.

quicklime 6y ago

> High dimensional spaces are unlikely to have local optima, and probably don’t have any optima at all. Can someone who knows more about DL than I do help me understand this a little better? The article uses the analogy of walls: > Just recall what is necessary for a set of parameters to be at a optimum. All the gradients need to be zero, and the hessian needs to be positive semidefinite. In other words, you need to be surrounded by walls. In 4 dimensions, you can walk through walls. GPT3 has 175 billion parameters. In 175 billion dimensions, walls are so far beneath your notice that if you observe them at all it is like God looking down upon individual protons. I'm struggling to understand what this really means in 4+ dimensions. But when I try to envision it going from 1 or 2 to 3 dimensions, it doesn't seem obvious at all that a 3D space should have fewer local optima than a 2D space. In fact, having a "universal function" like a deep network seems like it should have more local optima. What am I missing?

moultano 6y ago
Think of the model as a point in high d space that you're trying to trap inside a cage. (Corresponding to it being at a local minima and surrounded by higher points.) So you're trying to build a cube in n dimensions, which has 2n sides. Now imagine that each of those sides will randomly be there or not, probabilistically, with probability p. To actually trap a model like gpt3, p^175,000,000,000 has to be high enough that you observe a case during training where you roll a 1.
zachf 6y ago
The “physicist” explanation that I heard (meaning non-rigorous but good for building intuition) is that at every point where the derivative vanishes, for suitably random functions, every direction you move in will either be a direction where you increase or decrease at about 50-50 odds. In D dimensions there are 2D independent directions to rise or fall in (e.g. in D=2 dimensions, there’s north south east west), so there’s about 1/2^(2D) odds that any given critical point is actually a minimum if those probabilities are independent. That gets small really fast at large D. Obviously this is not rigorous, though.
- maxlamb 6y ago

unimpossible 6y ago

A lot of people talk about minima because thats the language we have for analyzing optimization techniques. Deep Learning is still new enough that there is lots of low hanging fruit to explore including empirical approaches and applying existing theoretical tools to try and explain DNNs. The community is slowly moving towards developing new tools specifically for deep learning properly analyze these networks and prove stuff (bounds, convergence etc) about them.

jostmey 6y ago

The author argues that deep-learning has abandoned statistics. I could not disagree more! Too much of the field was concerned with detailed proofs and mathematical formalism that were somewhat disconnected from probability theory. Modern machine learning (or AI or whatever) still has strong roots in probability and statistics. Loss functions are still based on concepts such as the log-likelihood function. Formal proofs and mathematics are essential, but can become a distraction from the end goal. It is like playing Chess by going after your opponents pawns instead of their king. I would say modern machine learning has become tantamount to experimental physics and this article is written from the perspective of a string theory theorist.

moultano 6y ago
That isn't what I'm arguing at all.
- candiodari 6y ago
  I think this is a way to really piss off statisticians: Stop talking about minima. Stop talking about how your optimization algorithm behaves around a minimum. Nobody ever trains their model remotely close to convergence. ADAM doesn’t even provably converge. All real models diverge! You are nowhere close to a minimum! Stop talking about minima already goddamnit! Why even think about minima?! Minima are a myth! Everybody proves their results for minima of a convex function. Actually arguing for divergent models. I mean, I love it, just because it's so very very cruel.

fizixer 6y ago

Woah! talk about sh-tting on deep learning and knowing jack about double descent, deep double descent. flag-worthy sh-tpost.

RandyRanderson 6y ago

Certainly a much shorter way to say this: if you have enough lines you can approximate any curve within a margin. This is what large neural networks are doing. Another way to look at it: most neural nets are just a bunch of polynomials stitched together. You can see this from the popularity of the relu activation function. when the relu is negative, that poly is always zero in that area. When positive it's some poly multiplied by a const - another poly. For nets that use other activation fns, they try to be linear in the area of most active input. So again they approximate a const * a ploy.

dougabug 6y ago
ReLU networks are piecewise linear.
- RandyRanderson 6y ago
  Have a look at some published nn architectures - you'll typically see a few nonlinear units/transforms and a LOT of ReLU ones.
blackbear_ 6y ago
You are missing the point. The surprising thing about deep learning is that it can generalize to unseen data so well. Polynomial regression cannot.
- mamon 6y ago
  If there is a pattern in data, such as it fitting on a curve, and you approximate that curve, then that should generalize to unseen data. What's surprising about that? A single polynominal regression wouldn't be able to do it, because some curves cannot be expressed as a polynominal, but superposition of multiple polynominals is apparently good enough.
  - moultano 6y ago
    If that worked, people would use it, but it doesn't, so they don't.

acadien 6y ago

Hey @moultano in response to your argument about walls and Nets not being in a minima, its my understanding nets always live on high dimensional saddle points and that's commonly referred to in literature. Even when you're optimizing you're just moving towards ever lower cost saddles that are closer to the optimum but almost never a local optimum (for the reasons spelled out in your post).

moultano 6y ago
Thank you. Several people have pointed that out, and I'm probably not reading the right papers. Is it common when people introduce a new flavor of adaptive SGD to address how it handles saddles specifically? It is probably just a a matter of what manages to bubble up to me rather than what work is actually getting done, but I felt like the non-convergence of ADAM got talked about a lot, but haven't seen people talking as much about how optimizers behave differently on the landscapes we actually observe.
- acadien 6y ago
  Saddles are a way of conceptualizing high dimensional optimization problems. If you have a 3 dimensional surface you can imagine a saddle as an isocurve that follows a minima in at least one dimension. Another way to conceptualize these is to think of being at the minima of a parabola in 2 dimensions, but then seeing you're not in a minima in a 3rd dimension. Any time you're in a minima in at least 1 dimension, you're on a saddle. You can extend this concept to a neural net which lives in millions of dimensions, undergoing SGD. When beginning an optimization run SGD moves in some direction to minimize the a bundled cost, inevitably stumbling into minima in (usually) many dimensions. Subsequent iterations will shift some dimensions out of minima and other dimensions into minima, the net is always living on a saddle during this process. There are many papers that discuss the process in these terms and others that implicitly use it. I wouldn't say its a "hot area of research" but more of a tool for thinking about these processes and sometimes gaining some insight in to why things get stuck during training.
muppet_frog 6y ago
This paper makes the points that it's the saddles and not local minima that are the problem: https://arxiv.org/abs/1406.2572 https://arxiv.org/abs/1406.2572 It was the basis for adding 'momentum' to optimizers - so that you could skate across the saddles.

21eleven 6y ago

I liked that part at the beginning where the author made clear they were going to discuss intuitions that, while they aren't proven, would be useful to make explicit for a more general audience. Good candor.

FriendlyNormie 6y ago
Good job repeating the second paragraph with slightly different words. What would we do without your brilliant contribution to the discussion?

blackbear_ 6y ago

There is another reason why training deep neural networks is not as difficult as it sounds: the landscape of the loss function seems to be made of broad "U"-shaped valleys that gently descend towards a small loss region. At initialization, the network is likely close to such valley, and once it gets there the rest of training is just a leisurely stroll. Formally, people have studied the spectrum of the Hessian and found that most of its eigenvectors are quite small with only a few, much larger ones. It all started with [1], with several recent extensions. [1] https://arxiv.org/pdf/1611.07476.pdf https://arxiv.org/pdf/1611.07476.pdf

aborsy 6y ago
This is just bunch of simulation, not a derivation.
- blackbear_ 6y ago
  True, but it has been confirmed experimentally several times. Unfortunately theoretical understanding in deep learning is quite lagging behind practical progress. This paper [1] seems to be getting closer to what you are looking for. [1] https://arxiv.org/abs/1910.02875 https://arxiv.org/abs/1910.02875
hammock 6y ago
Explain in layman's terms?
- macksd 6y ago
  Picture linear regression. If you have a bunch of data points you want to fit a line to, for any given line you can add up the vertical distance between all your data points and the line and come up with a measure of how inaccurate the line is. This is called your "loss". If you keep trying different values for the slope and intercept, you would find that this function is shaped like a big bowl, or valley. Regression is the process of repeatedly taking a step downhill until you can't go anywhere but up, and that must be the optimal line. Neural networks train in a similar way. You have a "loss" function that adds up how wrong your predictions are compared to the training data. You try different values for the weights in your neural network to see which ones send you downhill the fastest, step them in that direction, and repeat. Since the loss function in this case is more complex, it's not a single valley, but potentially many valleys. You can end up at a decent local minimum.

Schiphol 6y ago

I loved this post, thanks for writing it. I get the argument why one shouldn't expect local minima in very high dimensions. But then, what's wrong with the informal argument that there has to be a minimum because, well, the expected loss cannot be negative?

moultano 6y ago
With squared loss where it's easy for the loss to be zero, then yes, it will have lots of global minima, all at a loss of zero. For losses that asympotote, like logloss, they may have no minima.
- Schiphol 6y ago
  Thanks. I guess my worry was that, once you are doing extremely well and your loss is very low, gradients are no longer independent, and will tend to go mostly up. Is this wrong?
  - joshribakoff 6y ago
    I’m an ML newb but I think this would be true only of a converged model. Your model could always technically diverge in another epoch if learning rate is high enough and you process a batch of extreme outliers Even then, you may have still converged on a local optimum which was the take away I got from the article

woopwoop 6y ago

It seems to me all of these arguments apply just as well when the "deep" network has only one hidden layer.

moultano 6y ago
That's true. I left out saying that deeper networks represent a wider variety of functions more easily, because that seems generally intuitive to everyone. But the arguments about how easy it is to optimize them should apply equally well to wide and shallow networks as to very deep ones.
- woopwoop 6y ago
  Thanks for the response. Hmm, it's still pretty mysterious to me. Why should a deep network with the same number of parameters as a wide network represent a wider variety of functions? In some sense they represent the "same" number of functions, in the sense that the manifold of functions given by two network architectures with the same number of parameters have have the same dimension, even if one is wide while the other is deep.
  - bananaface 6y ago
    I think a deeper network has less degrees of freedom in which to move, or rather, in which to move usefully, because parameters are more interdependent. That means in order to generate a useful function, it has to learn more abstract features than a shallow and wide network. This is because any adjustment to irrelevant features that are unique to a small number of examples has a larger negative impact on the rest of the data than it would in a shallower net (except in the latter layers). Over time, abstract changes are stochastically rewarded and specific changes are penalised, at least for the earlier layers, and the latter layers then have to work with this more abstract information so they simply can't overfit that much. Would be interested in OP's take on this though.

brundolf 6y ago

Setting aside the primary subject, this is an excellent observation: > What I find however is that there are a base of unspoken intuitions that underlie expert understanding of a field, that are never directly stated in the literature, because they can’t be easily proved with the rigor that the literature demands. And as a result, the insights exist only in conversation and subtext, which make them inaccessible to the casual reader.

wenc 6y ago
This is very true. This is why conferences are important. A lot of knowledge is sociological in nature. Failures and tricks of the trade are discussed at the bar after 5. I've experienced this first hand. Academic publications and lectures, in their (rightful) pursuit of rigor, aren't usually the right space for conversations around hunches and experiences. Reputations are at stake, and most people are doing impression management. When experienced readers read a journal publication, they read it with the implicit understanding that it is a highly curated view of the messiness behind the scenes.
- brundolf 6y ago
  It seems toxic to me that there's no accepted public venue for that stuff, though. The fact that people are too afraid to relay certain useful information until they're tipsy; the emphasis on "doing impression management". Maybe it should remain separate from the rigorous stuff, but where's the "Op-Ed section" of academic publishing?
  - wenc 6y ago
    I feel the word "toxic" is maybe a bit too pejorative for what it is? Another framing is that it is "guild knowledge". The incentives in academia are complex and it affects how open some folks are or can afford to be with such knowledge. That said, there are certain open avenues for making such knowledge public. MathOverflow is one. Some academics document their guild knowledge in "technical papers" which they put up on their website. The ML community (which I'm not part of, but that I'm able to observe as an outsider) seems to be particularly open when it comes to publishing blog posts -- sometimes to gain reputational points? In some journals, arguments over publications are carried out in the Letters to the Editor section. Sometimes this leads to public feuds however, and some academic communities are small enough that if you make too many enemies your publications may be visited upon with disfavor when it comes time for peer review. It's not worth getting into public tiffs unless there's a principle at stake.

cs702 6y ago

TL;DR: For high-dimensional models (say, with millions to billions of parameters), there's always a good set parameters nearby, and when we start descending towards it, we are highly unlikely to get stuck, because almost always there's at least one path down along at least one among of all those dimensions -- i.e., there are no local optima. Once we've stumbled upon a good set of parameters, as measured by validation, we can stop. These intuitions are consistent with my experience... but I think there's more to deep learning. For instance, these intuitions fail to explain "weird" phenomena, such as "double descent" and "interpolation thresholds": * https://openai.com/blog/deep-double-descent/ https://openai.com/blog/deep-double-descent/ * https://arxiv.org/abs/1809.09349 https://arxiv.org/abs/1809.09349 * https://arxiv.org/abs/1812.11118 https://arxiv.org/abs/1812.11118 * See also: http://www.stat.cmu.edu/~ryantibs/papers/lsinter.pdf http://www.stat.cmu.edu/~ryantibs/papers/lsinter.pdf We still don't fully understand why stochastic gradient descent works so well in so many domains.

YeGoblynQueenne 6y ago

I find the article style unreadable. Could someone please say whether the author explains why deep learning shouldn't work? There is a bit at the start about how people in statistical learning throw their hands up at deep learning etc, but none of that makes sense to me. Neural nets are an idea as old as AI - even older, in fact. The need for deeper networks was well understood by the 1980's. There are well known results about feedforward neural nets with arbitrary hideen units being universal function approximators. Why shouldn't deep learning work?

moultano 6y ago
People coming from the perspectives of general optimization believed that they were impossible to train, due to being very non-convex. People coming from the perspective of classical statistics believed that they couldn't generalize due to needing large numbers of parameters. Both of those turned out to be very wrong, and this post is trying to explain why.
- YeGoblynQueenne 6y ago
  Thank you for the summarisation. Does the article give any examples of such arguments, or is it something stated as a commonly known fact? Coming from an artificial intelligence background I am not aware of such opinions. I know that deep neural nets were considered difficult to train until the re-discovery of backpropagation, but not because of anything to do with the shape of the error function. However, as usual there is confusion about what "generalisation" means. For example, I was in a summer school at Oxford a couple of years ago and one of the lectures made a similar point, about the surprising generalisation ability of deep neural nets. I approached the lecturer after the lecture and asked what they meant because the way I knew it, neural nets can't generalise, and they explained that they meant that they generalise surprisingly well on the test set but not on unseen data, or out-of-distribution data, i.e. not on any data that was not available to the researcher during training (as training, validation or test set). In other words, neural nets are great at "generalisation" in the sense of interpolation, but are almost completely incapable of "generalisation" in the form of extrapolation. I like to quote Francois Chollet of Keras on this: This stands in sharp contrast with what deep nets do, which I would call "local generalization": the mapping from inputs to outputs performed by deep nets quickly stops making sense if new inputs differ even slightly from what they saw at training time. Consider, for instance, the problem of learning the appropriate launch parameters to get a rocket to land on the moon. If you were to use a deep net for this task, whether training using supervised learning or reinforcement learning, you would need to feed it with thousands or even millions of launch trials, i.e. you would need to expose it to a dense sampling of the input space, in order to learn a reliable mapping from input space to output space. By contrast, humans can use their power of abstraction to come up with physical models—rocket science—and derive an exact solution that will get the rocket on the moon in just one or few trials. Similarly, if you developed a deep net controlling a human body, and wanted it to learn to safely navigate a city without getting hit by cars, the net would have to die many thousands of times in various situations until it could infer that cars and dangerous, and develop appropriate avoidance behaviors. Dropped into a new city, the net would have to relearn most of what it knows. On the other hand, humans are able to learn safe behaviors without having to die even once—again, thanks to their power of abstract modeling of hypothetical situations. https://blog.keras.io/the-limitations-of-deep-learning.html https://blog.keras.io/the-limitations-of-deep-learning.html In short, if the point of the article is that neural networks "work" because they generalise in the sense of extrapolation, then that's not right.

maxwells-daemon 6y ago

I don't think we can say for sure that early stopping is the main reason deep networks generalize. Double descent [1] shows that models continue to improve even once they've "interpolated" the training data (fit every point perfectly), and critical periods [2] suggest that the early part of training is responsible for most of the generalization performance even though much of the numerical improvement happens later. Overall it looks like gradient descent is a strong regularizer -- we know it tends to prefer small and low-variance weights, for example. So part of deep generalization has to do with how SGD is able to pick "good" features early, and then optimization pushes the unimportant weights to zero later (hence lottery tickets). [1] https://openai.com/blog/deep-double-descent/ https://openai.com/blog/deep-double-descent/ and other papers. [2] https://arxiv.org/abs/1711.08856 https://arxiv.org/abs/1711.08856 and others.

owenshen24 6y ago
To add some more context, here's a rather readable summary: https://www.greaterwrong.com/posts/FRv7ryoqtvSuqBxuT/understanding-deep-double-descent https://www.greaterwrong.com/posts/FRv7ryoqtvSuqBxuT/underst...
machinelearning 6y ago
I think you misunderstood the point of deep double descent. The x-axis is not number of training epochs, it is model capacity. I think you'd be interested in https://arxiv.org/abs/1611.03530 https://arxiv.org/abs/1611.03530. It discusses how SGD is an implicit regularizer. We also actually want high variance weights for symmetry breaking.

Animats 6y ago

Now that's fascinating. I'd thought of machine learning as a form of optimization. Things like support vector machines really were hill climbing for some kind of local optimum point. But at a billion dimensions, you're doing something else entirely. I once went through Andrew Ng's old machine learning course on video, and he was definitely doing optimization. The last time I actually had to do numerical optimization using gradients, I was trying to solve nonlinear differential equations for a physics engine in about a 20-dimensional space of joint angles that was very "stiff". That is, some dimensions might be many orders of magnitude steeper than another. It's like walking on a narrow zigzagging mountain ridge without falling off. So deep learning is not at all like either of those. Hm.

mrfusion 6y ago

This quote: there are a base of unspoken intuitions that underlie expert understanding of a field, that are never directly stated in the literature, because they can’t be easily proved with the rigor that the literature demands. And as a result, the insights exist only in conversation and subtext, which make them inaccessible to the casual reader. I’d love to hear these intuitions from every field. Anyone got some?

Fordec 6y ago
All sensor data, the closer you get to the analog side of things, is bullshit. It's just about smoothing over the bullshit enough to make the tolerances workable for real world applications. We call this bullshit smoothing "calibration". If you're doing work on sensor data and don't have every calibration parameter, whether from configuration or magic factory numbers and statistical tolerances, someone, somewhere is pulling the wool over the eyes of the software guy downstream that works with the final data. Ever looked at weather data from two separate apps and have values vary by multiple degrees? Two different pipeline just sprinkled on their own versions of interpretations on top of raw sewage data.
- brandmeyer 6y ago
  > Ever looked at weather data from two separate apps and have values vary by multiple degrees? Actually this is because surface temperatures really do have this kind of variation.

ummonk 6y ago

Great article, though I never understood why people would think deep learning shouldn't work.

deleted 6y ago

[deleted]

KKKKkkkk1 6y ago

But it doesn't. Researchers have been saying for several years now that computer vision is more accurate than human vision, and face recognition was one of the first problems "solved." And yet when the pandemic hit, Apple had to scramble to adjust its unlock mechanism in iOS 13.5 because Face ID cannot recognize people wearing masks [1]. Humans have no trouble identifying people wearing masks. We are now almost a year into the pandemic, iOS 14 has been released, Face ID has not been fixed, and now we hear that Apple is bringing Touch ID back [2]. So sure, you've developed a methodology that can overfit nicely not only the train data but even the test data. But it still fails miserably when you apply your model in the field. [1] https://www.theverge.com/2020/5/20/21265019/apple-ios-13-5-out-now-unlock-iphone-face-mask-id-exposure-notification-covid-19 https://www.theverge.com/2020/5/20/21265019/apple-ios-13-5-o... [2] https://appleinsider.com/articles/20/10/16/under-display-touch-id-on-an-iphone-is-still-coming-leaker-claims https://appleinsider.com/articles/20/10/16/under-display-tou...

why_only_15 6y ago
I don't think this is quite fair. Of course a machine learning model doesn't work on things it wasn't trained for. Computer vision is not more accurate than human vision in every possible field. And are you really sure humans have no trouble identifying people wearing masks? Lots of people look pretty similar, especially if all you can see is their face.
rajansaini 6y ago
Intuitively, it seems like voice signatures, body language (like walking habits), and height/weight would play a larger role in helping humans identify a masked person than exposed facial features. Because the network behind Apple's facial recognition software cannot have access to this kind of data (well, maybe voice, but that doesn't seem secure), I'm not sure this is a fair comparison. Would love to be refuted, however.
- teorema 6y ago
  I don't know that I'm looking to refute you per se but... A better example than face masks maybe is the recent controversy over Twitter's AI and Obama images (https://www.theverge.com/2020/9/20/21447998/twitter-photo-preview-white-black-faces https://www.theverge.com/2020/9/20/21447998/twitter-photo-pr...). A lot was made of racial issues, which is fine, but the broader issue is why subtle changes in photos, like cropping, should confuse things so completely. The target piece (the focus of this HN thread) sums itself up this way: "There’s a good set of params somewhere nearby. When we start walking to it, we can’t ever get stuck along the way, because there are no local optima. Once we’ve stumbled upon a good set of parameters, we’ll know it and we can just stop." I think there's some useful insights there, but this is in many ways the definition of local optima. What I might argue is that because there's so many locations in high-dimensional space that will satisfy some classification goal, it's "easy" to find one that works with regard to some population that defines the model development space (training + test). However, that model development space/population is itself implicitly defined by a certain set of constraints -- it's a subpopulation of some broader population. What you want to generalize to to define overfitting is broader. You can still not overfit to your model development space, but be overfitting with regard to some broader set of possible inputs. Whether or not the constraints of the model development population/space are important and reasonable considerations -- e.g., in your argument, not having access to things like body language etc -- is maybe a little variable. In some cases the implicit defining characteristics of the model development population are meaningful, but in other cases they're hidden. In Twitter's case, you end up finding out later that there's weird things that probably defined the space of their images that they didn't intend. It's only in the adversarial case that you learn about this. In classical statistics, you talk about generalization and overfitting, but there's an implicit population you're sampling from that defines both of those things. That is, you have a training/fitting/initial sample, and you ask yourself how well your model would perform on a test/validation sample. But implicit in that is some assumption about what it means to be a random sample from the same population. I think lots of times with DL, the cross-validation/test sample is also implicitly defined as coming from some population. But the population isn't some model, it's some source. Some image dataset, something like that. There will be things about that source that are "of interest", but other things that are idiosyncratic about it, and unrepresentative of the "real" population of interest. In this way, I'm not sure that held-out samples from some source are really the right way to think of generalization and overfitting -- I think the adversarial setting is the generalization setting. https://www.sciencemag.org/news/2020/05/eye-catching-advances-some-ai-fields-are-not-real https://www.sciencemag.org/news/2020/05/eye-catching-advance... Along the way from the classical to the DL regime I think there have been some overlooked issues about what it means to generalize, what you're really sampling from, and what your "population" actually is. It parallels tensions about theory versus experimentation because having a population in the classical case that you're sampling from requires a certain data-generating theory, which is often lacking in DL. The closest thing in the classical regime to DL generalization theory is maybe a sort of ultra-high-dimensional bootstrapping with random effects: showing that your bootstrap samples are close to your observed sample isn't the same thing as showing they're close to the population, or to other samples drawn from that population, especially in the presence of random effects.

hexo 6y ago

Blank page? :(

foxes 6y ago

There is a trick in physics I am reminded about. In infinite dimensions there is no way to have a Gaussian measure on just an infinite dimensional Hilbert space. It needs to be embedded inside a bigger infinite dimensional space and then you can have some relative measure. So you do not look at just a Gaussian integral, you look at a quotient of Gaussian integrals. Perhaps there is a similar idea. Perhaps there is some sort of renormalisation that would make neural networks work better. Even if your neural network is infinite dimensional it still makes sense to talk about some surface relatively.

ZeljkoS 6y ago

It looks like the article was deleted ("Oops! That page can’t be found"). Here is the Google Cache: https://webcache.googleusercontent.com/search?q=cache:5HMZ_ZO0mwkJ:https://moultano.wordpress.com/2020/10/18/why-deep-learning-works-even-though-it-shouldnt/+&cd=1&hl=en&ct=clnk&gl=hr https://webcache.googleusercontent.com/search?q=cache:5HMZ_Z...

turingbook 6y ago

The article was gone?

mikhailfranco 6y ago

Another interesting explanation for deep learning's success in the physical world: Why does deep and cheap learning work so well? Max Tegmark et al., https://arxiv.org/abs/1608.08225 https://arxiv.org/abs/1608.08225