5 ms·

I'm a self-described Bayesian* at my day job, but the author needs to do better to convince me that the Bayesian approach is worth it in the deep learning space

I'm a self-described Bayesian* at my day job, but the author needs to do better to convince me that the Bayesian approach is worth it in the deep learning space. As far as I can tell, deep learning folks don't give two shits about uncertainty intervals, much less marginalization. All that matters is minimizing that test error as fast as possible. So what if you get a posterior for each parameter... Who cares about the parameters in a neural network as long as the predictions seem well calibrated? The most convincing rationale for adopting a Bayesian perspective is contained the collected works of Jim Berger, which I see is cited by the author... but not used in the manuscript. * Of course, a Bayesian is just a statistician that uses Bayesian techniques even when it's not appropriate -- Andrew Gelman

MiroF 7y ago
I agree - far too often I just see "safety critical applications" thrown around and that is definitely not enough of a justification in my mind. There are alternative ways of getting uncertainty estimates that are far less work.
ssivark 7y ago
> As far as I can tell, deep learning folks don't give two shits about uncertainty intervals, much less marginalization. All that matters is minimizing that test error as fast as possible. So what if you get a posterior for each parameter... Who cares about the parameters in a neural network as long as the predictions seem well calibrated? Well, the one thing that they ought to actually give a shit about is generalization. All else (Bayesian or not) is only in service of that. Deep learning systems require too much data (and often not raw data), crazy amounts of compute, and still fail under comically irrelevant tweaks of the problem (eg: adversarial noise). Yes, the PAC learning model starts off by assuming that the test set had the same “data distribution” as the train set. It’s obviously not exactly the same (otherwise learning is kinda irrelevant), but deep learning seems incredibly fragile in its ability to generalize. It seems to require crazy amounts of domain randomization during training to avoid overfitting. Eg: The recent OpenAI Rubik’s cube demonstration (while an impressive achievement) was an indication of the fundamental silliness of deep RL (at least as of now). To solve the problem, they had to use classical Rubik’s cube solving algorithms, and completely model the physics of the situation (including friction on the surfaces!) and use that model to generate training data while varying every parameter for domain randomization. All that structured prior knowledge is absolutely unfeasible for even a slightly more complicated situation. And after all that, the robot had something like a 20% success rate IIRC. Impressive work on the team’s part, no doubt... but the conceptual framework does not inspire confidence.
- mjburgess 7y ago
  I doubt that limits to generalizing NNs will be solved by mere bayesianfication. NNs poorly generalize, as with all of ML, because the strategy is the modelling of the data, not of the world. Since data stands in as proxy for the environment, a NN will only generalize insofar as it has the data to do so. There is no 3rd dimension in an infinite number of 2D photographs -- an no "bayesianification" will put it there.
  - jacquesm 7y ago
    > because the strategy is the modelling of the data, not of the world. But that is how all learning works. Even the most advanced physics models are verified using experimental data. If data is found that contradicts the model the model is deemed faulty and will be adjusted to match the insights gained from the new data. DL just automates that approach aiming for the minimum gap between 'theory' and 'data' where the theory is expressed as a number of weights, thus encoding 'insight' in a - for now - non-reversible way. Just like wetware typically has not clue how it arrived at a certain conclusion, we use the words intuition and synonyms when actually we mean 'guess based on past experience'.
    - tsimionescu 7y ago
      That is not how human learning works. Manuals don't contain hundreds of thousands of examples to allow students to draw their own conclusion. And for inventing new physics, there are two very different approaches. One is perhaps similar to machine learning, where people look at an existing model, find experimental data that doesn't match the model, and adjust the model slightly until it matches the new data. But another mode is where you have a model with certain mathematical properties, then you discover new mathatics that give more properties, and you only then ask for more data to check these extensions.
      - Schiphol 7y ago
        How does human learning work, then? Do you have pointers to literature on alternative formal learning theories? I am genuinely curious :)
lostdog 7y ago
> As far as I can tell, deep learning folks don't give two shits about uncertainty intervals, much less marginalization. Maybe folks making snapchat filters don't care, but this is absolutely vital if you're doing something with a low margin of error (self-driving cars, financial work, etc...). If your neural net can tell you if it thinks it could be making an error, that information is absolutely vital for keeping your system from messing up big time. The problem is that none of the Bayesian methods for machine learning work well and quickly (please do correct me if I'm wrong--I will become my boss's favorite person). If they did, many many practitioners would be very excited to use them.
- jacquesm 7y ago
  Bayesian methods work reasonably well as long as the parameter space is relatively limited and you have enough data to extract some advantage but not enough to power a deep learning model. You also usually do not rely on Bayesian methods exclusively to solve some problem but use it in conjunction with other methods (sanity checks) to ensure you aren't outputting nonsense.
  - MiroF 7y ago
    Ah yes, the well-known limited parameter space of deep neural networks
- eli_gottlieb 7y ago
  >The problem is that none of the Bayesian methods for machine learning work well and quickly (please do correct me if I'm wrong--I will become my boss's favorite person). If they did, many many practitioners would be very excited to use them. There are loads of methods based on variational autoencoders that seem to work fairly well for various problems.
- nestorD 7y ago
  After a bit of search I found something that appears to work both well and quickly: http://www.cs.ox.ac.uk/people/yarin.gal/website/blog_3d801aa532c1ce.html http://www.cs.ox.ac.uk/people/yarin.gal/website/blog_3d801aa... The basic idea is brilliant: take a deep neural network with dropout, keep the dropout at evaluation time, run it several times and deduce the uncertainty from the observed variance (with a bit of math). This lets you add an uncertainty estimation to the output of any deep neural network that uses dropout.
Fomite 7y ago
"...deep learning folks don't give two shits about uncertainty intervals..." This is a failing of those folks for many applications.
commanderdata87 7y ago
Read the note carefully: "While improvements in calibration are an empirically recognized benefit of a Bayesian approach, the enormous potential for gains in accuracy through Bayesian marginalization with neural networks is a largely overlooked advantage." Because the different parameter settings correspond to different high performing models, averaging their predictions will lead to more accurate predictions.
- jacquesm 7y ago
  Averaging predictions can lead to improvements but this isn't necessarily so. It may also result in simply dragging down the accuracy in the direction of the lesser method if it is consistently less good than the others in the batch. Averaging works well when multiple methods tend towards different outliers, that way you end up cancelling some of that out.
- MiroF 7y ago
  I have yet to see someone successfully produce a high dimensional posterior over all of the parameters in the neural network. Usually, people make the simplifying assumption of different posteriors for each parameter, which means that you can't just extract a bunch of different models.
mratsim 7y ago
Uncertainty is absolutely key everywhere decision making has an impact on human lives. Detecting if we see an expresso or a cappucino, sure fine. But uncertainty on a CT scan would be extremely valuable just to avoid people (even trained professionals) falling into the common bias of trusting the output. Say the model outputs 90% chance of nothing, 10% chance of a deadly condition that needs to be investigated ASAP. Doctors are often overworked and tired and can miss it. Knowing where the certainties and uncertainties of the model lies would allow the doctor to compare that to its own training "Ah but it didn't at all judge based on X and Y" and limit decision machine based on "trusting the algorithms". This is a similar problem that rating system based on just a number of stars are facing, you need to read the comments and the why just to filter out astroturfing. Coming back to ML, this is why I think more emphasis should be given to LIME [1], SHAP [2] and also Microsoft's Explainable Boosting Machine [3] which ironically is not a gradient boosting algorithm but a Generalized Additive Model (i.e. like GLM but Squared instead of Linear) [1] https://github.com/marcotcr/lime https://github.com/marcotcr/lime [2] https://github.com/slundberg/shap https://github.com/slundberg/shap [3] https://github.com/interpretml/interpret https://github.com/interpretml/interpret
- throwlaplace 7y ago
  i have a decent grasp of stats (having read all of casella berger) and i don't understand how you can truly quantify uncertainty since confidence intervals or pvalues assume a population distribution. i feel like it's a shell game. given a uniform prior what can you tell me about the uncertainty? and if you've chosen some other prior how can you qualify that decision? this is not even considering that if you take it all the way down to first principles, there isn't even such a thing as a continuous prior or one with uncountable support.
  - Mooks79 7y ago
    This is an area where Bayesian methods really come to the fore - dealing with non-normal, even non-parametric, distributions. That’s not to say they can’t be treated in frequentists stats. I’ve never read that book but perhaps it stops before dealing with how to handle all that stuff. Regarding your specific point about how do you quantify error regarding your chosen prior? The answer in a Bayesian framework lies in things like credibility intervals, posterior distributions, posterior predictive distributions, depending on what you’re actually trying to quantify. Very long story short, the latter allows you to sample from a distribution of predictions - for which you can then form a variety of error estimates (and the distribution doesn’t have to be parametric). That will quantify uncertainty for the model you’ve made using the prior you chose (and the data). But usually we wouldn’t recommend changing your prior just to get lower uncertainty unless you have some rationality behind changing the prior. Priors should really be chosen based on domain knowledge etc etc etc.
zachwill 7y ago
Where's the best place to read through the collected works of Jim Berger? (Thanks!)

MiroF 7y ago

I agree - far too often I just see "safety critical applications" thrown around and that is definitely not enough of a justification in my mind. There are alternative ways of getting uncertainty estimates that are far less work.

ssivark 7y ago

> As far as I can tell, deep learning folks don't give two shits about uncertainty intervals, much less marginalization. All that matters is minimizing that test error as fast as possible. So what if you get a posterior for each parameter... Who cares about the parameters in a neural network as long as the predictions seem well calibrated? Well, the one thing that they ought to actually give a shit about is generalization. All else (Bayesian or not) is only in service of that. Deep learning systems require too much data (and often not raw data), crazy amounts of compute, and still fail under comically irrelevant tweaks of the problem (eg: adversarial noise). Yes, the PAC learning model starts off by assuming that the test set had the same “data distribution” as the train set. It’s obviously not exactly the same (otherwise learning is kinda irrelevant), but deep learning seems incredibly fragile in its ability to generalize. It seems to require crazy amounts of domain randomization during training to avoid overfitting. Eg: The recent OpenAI Rubik’s cube demonstration (while an impressive achievement) was an indication of the fundamental silliness of deep RL (at least as of now). To solve the problem, they had to use classical Rubik’s cube solving algorithms, and completely model the physics of the situation (including friction on the surfaces!) and use that model to generate training data while varying every parameter for domain randomization. All that structured prior knowledge is absolutely unfeasible for even a slightly more complicated situation. And after all that, the robot had something like a 20% success rate IIRC. Impressive work on the team’s part, no doubt... but the conceptual framework does not inspire confidence.

mjburgess 7y ago
I doubt that limits to generalizing NNs will be solved by mere bayesianfication. NNs poorly generalize, as with all of ML, because the strategy is the modelling of the data, not of the world. Since data stands in as proxy for the environment, a NN will only generalize insofar as it has the data to do so. There is no 3rd dimension in an infinite number of 2D photographs -- an no "bayesianification" will put it there.
- jacquesm 7y ago
  > because the strategy is the modelling of the data, not of the world. But that is how all learning works. Even the most advanced physics models are verified using experimental data. If data is found that contradicts the model the model is deemed faulty and will be adjusted to match the insights gained from the new data. DL just automates that approach aiming for the minimum gap between 'theory' and 'data' where the theory is expressed as a number of weights, thus encoding 'insight' in a - for now - non-reversible way. Just like wetware typically has not clue how it arrived at a certain conclusion, we use the words intuition and synonyms when actually we mean 'guess based on past experience'.
  - tsimionescu 7y ago
    That is not how human learning works. Manuals don't contain hundreds of thousands of examples to allow students to draw their own conclusion. And for inventing new physics, there are two very different approaches. One is perhaps similar to machine learning, where people look at an existing model, find experimental data that doesn't match the model, and adjust the model slightly until it matches the new data. But another mode is where you have a model with certain mathematical properties, then you discover new mathatics that give more properties, and you only then ask for more data to check these extensions.
    - Schiphol 7y ago
      How does human learning work, then? Do you have pointers to literature on alternative formal learning theories? I am genuinely curious :)

lostdog 7y ago

> As far as I can tell, deep learning folks don't give two shits about uncertainty intervals, much less marginalization. Maybe folks making snapchat filters don't care, but this is absolutely vital if you're doing something with a low margin of error (self-driving cars, financial work, etc...). If your neural net can tell you if it thinks it could be making an error, that information is absolutely vital for keeping your system from messing up big time. The problem is that none of the Bayesian methods for machine learning work well and quickly (please do correct me if I'm wrong--I will become my boss's favorite person). If they did, many many practitioners would be very excited to use them.

jacquesm 7y ago
Bayesian methods work reasonably well as long as the parameter space is relatively limited and you have enough data to extract some advantage but not enough to power a deep learning model. You also usually do not rely on Bayesian methods exclusively to solve some problem but use it in conjunction with other methods (sanity checks) to ensure you aren't outputting nonsense.
- MiroF 7y ago
  Ah yes, the well-known limited parameter space of deep neural networks
eli_gottlieb 7y ago
>The problem is that none of the Bayesian methods for machine learning work well and quickly (please do correct me if I'm wrong--I will become my boss's favorite person). If they did, many many practitioners would be very excited to use them. There are loads of methods based on variational autoencoders that seem to work fairly well for various problems.
nestorD 7y ago
After a bit of search I found something that appears to work both well and quickly: http://www.cs.ox.ac.uk/people/yarin.gal/website/blog_3d801aa532c1ce.html http://www.cs.ox.ac.uk/people/yarin.gal/website/blog_3d801aa... The basic idea is brilliant: take a deep neural network with dropout, keep the dropout at evaluation time, run it several times and deduce the uncertainty from the observed variance (with a bit of math). This lets you add an uncertainty estimation to the output of any deep neural network that uses dropout.

Fomite 7y ago

"...deep learning folks don't give two shits about uncertainty intervals..." This is a failing of those folks for many applications.

commanderdata87 7y ago

Read the note carefully: "While improvements in calibration are an empirically recognized benefit of a Bayesian approach, the enormous potential for gains in accuracy through Bayesian marginalization with neural networks is a largely overlooked advantage." Because the different parameter settings correspond to different high performing models, averaging their predictions will lead to more accurate predictions.

jacquesm 7y ago
Averaging predictions can lead to improvements but this isn't necessarily so. It may also result in simply dragging down the accuracy in the direction of the lesser method if it is consistently less good than the others in the batch. Averaging works well when multiple methods tend towards different outliers, that way you end up cancelling some of that out.
MiroF 7y ago
I have yet to see someone successfully produce a high dimensional posterior over all of the parameters in the neural network. Usually, people make the simplifying assumption of different posteriors for each parameter, which means that you can't just extract a bunch of different models.

mratsim 7y ago

Uncertainty is absolutely key everywhere decision making has an impact on human lives. Detecting if we see an expresso or a cappucino, sure fine. But uncertainty on a CT scan would be extremely valuable just to avoid people (even trained professionals) falling into the common bias of trusting the output. Say the model outputs 90% chance of nothing, 10% chance of a deadly condition that needs to be investigated ASAP. Doctors are often overworked and tired and can miss it. Knowing where the certainties and uncertainties of the model lies would allow the doctor to compare that to its own training "Ah but it didn't at all judge based on X and Y" and limit decision machine based on "trusting the algorithms". This is a similar problem that rating system based on just a number of stars are facing, you need to read the comments and the why just to filter out astroturfing. Coming back to ML, this is why I think more emphasis should be given to LIME [1], SHAP [2] and also Microsoft's Explainable Boosting Machine [3] which ironically is not a gradient boosting algorithm but a Generalized Additive Model (i.e. like GLM but Squared instead of Linear) [1] https://github.com/marcotcr/lime https://github.com/marcotcr/lime [2] https://github.com/slundberg/shap https://github.com/slundberg/shap [3] https://github.com/interpretml/interpret https://github.com/interpretml/interpret

throwlaplace 7y ago
i have a decent grasp of stats (having read all of casella berger) and i don't understand how you can truly quantify uncertainty since confidence intervals or pvalues assume a population distribution. i feel like it's a shell game. given a uniform prior what can you tell me about the uncertainty? and if you've chosen some other prior how can you qualify that decision? this is not even considering that if you take it all the way down to first principles, there isn't even such a thing as a continuous prior or one with uncountable support.
- Mooks79 7y ago
  This is an area where Bayesian methods really come to the fore - dealing with non-normal, even non-parametric, distributions. That’s not to say they can’t be treated in frequentists stats. I’ve never read that book but perhaps it stops before dealing with how to handle all that stuff. Regarding your specific point about how do you quantify error regarding your chosen prior? The answer in a Bayesian framework lies in things like credibility intervals, posterior distributions, posterior predictive distributions, depending on what you’re actually trying to quantify. Very long story short, the latter allows you to sample from a distribution of predictions - for which you can then form a variety of error estimates (and the distribution doesn’t have to be parametric). That will quantify uncertainty for the model you’ve made using the prior you chose (and the data). But usually we wouldn’t recommend changing your prior just to get lower uncertainty unless you have some rationality behind changing the prior. Priors should really be chosen based on domain knowledge etc etc etc.

zachwill 7y ago

Where's the best place to read through the collected works of Jim Berger? (Thanks!)