3 ms·

Student enrolled in his Bayesian Machine Learning class ORIE 6741. The formulation of adding bayesian inference to deep learning can be applied to CNN's as well

Student enrolled in his Bayesian Machine Learning class ORIE 6741. The formulation of adding bayesian inference to deep learning can be applied to CNN's as well. I plan on using his work on Stochastic Gradient Hamiltonian Monte Carlo to do Bayesian Inference on Image Segmentation as well. For more interest, check out this blog, super interesting: http://www.cs.ox.ac.uk/people/yarin.gal/website/blog_3d801aa532c1ce.html http://www.cs.ox.ac.uk/people/yarin.gal/website/blog_3d801aa...

MrQuincle 9y ago
Did you read "Variational Gaussian Dropout is not Bayesian", https://arxiv.org/abs/1711.02989 https://arxiv.org/abs/1711.02989? https://twitter.com/dustinvtran/status/928493076854669312 https://twitter.com/dustinvtran/status/928493076854669312
- elcritch 9y ago
  I posted before refreshing and missed the link to this paper. Will need to read through it.
tMcGrath 9y ago
I'd like to flag up this paper on HMC and minibatch inference: https://arxiv.org/abs/1502.01510v1 https://arxiv.org/abs/1502.01510v1. It seems that SGHMC might not always behave as well as you might like. I should say this isn't to pour cold water on the idea of Bayesian deep learning - I think it's an extremely sensible idea and undoubtedly a good direction for research, but just wanted to caution that this particular MCMC method might have subtle flaws.
- eref 9y ago
  I wished more academic papers would have a "Limitations" section. Often it is so hard to tell what these are, e.g.: Does it scale well? How much faster/slower is it than other methods? Does it converge reliably?
  - pishpash 9y ago
    No incentive for such a thing.
    - eref 9y ago
      Neither is there for reproducible code, but now there are artificial incentives such as the Reproduction Challenge.
elcritch 9y ago
Interesting blog post! From a 10,000 ft overview is this method utilizing the conditional posterior's to incorporate uncertainty information into the model GAN model? If so how does it differ from using drop-out's to approximate Bayesian behavior ala https://arxiv.org/pdf/1506.02142.pdf https://arxiv.org/pdf/1506.02142.pdf ?

MrQuincle 9y ago

Did you read "Variational Gaussian Dropout is not Bayesian", https://arxiv.org/abs/1711.02989 https://arxiv.org/abs/1711.02989? https://twitter.com/dustinvtran/status/928493076854669312 https://twitter.com/dustinvtran/status/928493076854669312

elcritch 9y ago
I posted before refreshing and missed the link to this paper. Will need to read through it.

tMcGrath 9y ago

I'd like to flag up this paper on HMC and minibatch inference: https://arxiv.org/abs/1502.01510v1 https://arxiv.org/abs/1502.01510v1. It seems that SGHMC might not always behave as well as you might like. I should say this isn't to pour cold water on the idea of Bayesian deep learning - I think it's an extremely sensible idea and undoubtedly a good direction for research, but just wanted to caution that this particular MCMC method might have subtle flaws.

eref 9y ago
I wished more academic papers would have a "Limitations" section. Often it is so hard to tell what these are, e.g.: Does it scale well? How much faster/slower is it than other methods? Does it converge reliably?
- pishpash 9y ago
  No incentive for such a thing.
  - eref 9y ago
    Neither is there for reproducible code, but now there are artificial incentives such as the Reproduction Challenge.

elcritch 9y ago

Interesting blog post! From a 10,000 ft overview is this method utilizing the conditional posterior's to incorporate uncertainty information into the model GAN model? If so how does it differ from using drop-out's to approximate Bayesian behavior ala https://arxiv.org/pdf/1506.02142.pdf https://arxiv.org/pdf/1506.02142.pdf ?