3 ms·

Bayesian can be seen as a subset of deep learning or hell a superset. AI is a superset and Machine learning is a subset of AI and most funding is in deep learn

Bayesian can be seen as a subset of deep learning or hell a superset. AI is a superset and Machine learning is a subset of AI and most funding is in deep learning. Once Deep Learning hit the limit I believe there will be an AI winter. Maybe there will be hype around statistic (cross fingers) which will lead to Bayesian and such.

johnmoberg 8y ago
How can Bayesian stuff be seen as a subset or superset of deep learning?
- kamaal 8y ago
  I guess the point that digitalzombie is trying to make is most of what we call AI or ML or even Deep learning is simply extension of statistics on computers. Things like the German tank problem or the problem of hardening airplanes during WW2 have that very AI'esque feel to it. Where you use data to build a model, then let that data from the model to change the model as it fits. Also the whole thing about 'decision making' is either bayesian or frequency based models in nature. Most of these algorithms and math has long existed before the current boom. Its just that the raw computing power and resources that you have today make it possible for you to deal with large amounts of data to stress test your models.
eli_gottlieb 8y ago
>Bayesian can be seen as a subset of deep learning or hell a superset. eh-hem DIE, HERETIC! eh-hem Ok, with that out of my system, no, Bayesian methods are definitely not a subset of deep learning, in any way. Hierarchical Bayes could be labeled "deep Bayesian methods" if we're marketing jerks, but Bayesian methods mostly do not involve neural networks with >3 hidden layers. It's just a different paradigm of statistics.
- digitalzombie 8y ago
  My mentor was very very adamant about Bayesian network and hierarchical as being deep learning. He sees the latent layer in the hierarchical model as the hidden layer and the Bayesian just have a strict restrictions/assumptions to the network where as the deep learning is more dumb and less assuming. A few of my professor thinks that PGM, probability graphical model is a super set of deep learning/neural network. This is where my thinking come from. IIRC, a paper have shown that gradient descent seems to exhibit MCMCs (blog with paper link inside that led to this conclusion of mine: http://www.inference.vc/everything-that-works-works-because-its-bayesian-2/ http://www.inference.vc/everything-that-works-works-because-...). But I am not an expert in Neural Network nor know the topic well enough to say such a thing. Other than was deferring to opinions of some one that's better than myself. So I'll keep this in mind and hopefully one day have the time to do more research into this topic. Thank you.
  - eli_gottlieb 8y ago
    I think your link, and your mentor, are somewhat fundamentalist about their Bayesianism.

johnmoberg 8y ago

How can Bayesian stuff be seen as a subset or superset of deep learning?

kamaal 8y ago
I guess the point that digitalzombie is trying to make is most of what we call AI or ML or even Deep learning is simply extension of statistics on computers. Things like the German tank problem or the problem of hardening airplanes during WW2 have that very AI'esque feel to it. Where you use data to build a model, then let that data from the model to change the model as it fits. Also the whole thing about 'decision making' is either bayesian or frequency based models in nature. Most of these algorithms and math has long existed before the current boom. Its just that the raw computing power and resources that you have today make it possible for you to deal with large amounts of data to stress test your models.

eli_gottlieb 8y ago

>Bayesian can be seen as a subset of deep learning or hell a superset. eh-hem DIE, HERETIC! eh-hem Ok, with that out of my system, no, Bayesian methods are definitely not a subset of deep learning, in any way. Hierarchical Bayes could be labeled "deep Bayesian methods" if we're marketing jerks, but Bayesian methods mostly do not involve neural networks with >3 hidden layers. It's just a different paradigm of statistics.

digitalzombie 8y ago
My mentor was very very adamant about Bayesian network and hierarchical as being deep learning. He sees the latent layer in the hierarchical model as the hidden layer and the Bayesian just have a strict restrictions/assumptions to the network where as the deep learning is more dumb and less assuming. A few of my professor thinks that PGM, probability graphical model is a super set of deep learning/neural network. This is where my thinking come from. IIRC, a paper have shown that gradient descent seems to exhibit MCMCs (blog with paper link inside that led to this conclusion of mine: http://www.inference.vc/everything-that-works-works-because-its-bayesian-2/ http://www.inference.vc/everything-that-works-works-because-...). But I am not an expert in Neural Network nor know the topic well enough to say such a thing. Other than was deferring to opinions of some one that's better than myself. So I'll keep this in mind and hopefully one day have the time to do more research into this topic. Thank you.
- eli_gottlieb 8y ago
  I think your link, and your mentor, are somewhat fundamentalist about their Bayesianism.