6 ms·
A Gentle Introduction to Bayes’ Theorem for Machine Learning
- WilliamEdward 7y agoAlways nice to see a little piece of python code show up whenever one of these ML blogs get posted to HN. It's becoming, or has become, the lingua franca of this field.
- usmannk 7y agoI'm very excited for Tensorflow for Swift, personally. The type system will make development so much more pleasant IMO. Judging by the way things have been lately though, I imagine that by the time TF for Swift is ready I'll be reaching for PyTorch at every opportunity anyways...
- arman_ashrafian 7y agoI think the fact that Swift has first-class automatic differentiation is an even bigger deal than the fact that it is strongly-typed. Here is an interesting write-up about it... https://gist.github.com/rxwei/30ba75ce092ab3b0dce4bde1fc2c9f1d https://gist.github.com/rxwei/30ba75ce092ab3b0dce4bde1fc2c9f...
- hnhg 7y agoThis is a far gentler introduction, and the rest of the blog is pretty good too: https://www.countbayesie.com/blog/2016/5/1/a-guide-to-bayesian-statistics https://www.countbayesie.com/blog/2016/5/1/a-guide-to-bayesi... Edit: this is a different take on the subject but an enjoyable and accessible read too: http://mbmlbook.com/toc.html http://mbmlbook.com/toc.html
- mlevental 7y agochange my mind: bayes in practice is just a way to regularize your model and the language of bayes makes it seem principled but really you could use literally any regularizer and it would work almost just as well. i believe this because ultimately you're always going to minimize loglikelihood anyway (and so the prior becomes the regularization term).
- rwilson4 7y agoCounterpoint: regularization is just a way of specifying a Bayesian prior for maximum a posteriori estimation.
- mlevental 7y agobut what value is that perspective? how do i use this to actually fit a model?
- salty_biscuits 7y agoWell you could draw samples of the parameters with a Bayesian setup via MCMC or get a distribution over them via a variational approximation, rather than getting some sort of maximum likelihood (MAP whatever) value for the parameters of the model via solving an optimization problem. This seems much more general (and practically useful). So I think it is the other way around regularizers are just priors (that you arrived at somehow).
- Double_Org 7y agoA bottling company is interested in determining the accuracy with which their equipment is filling bottles of water. One answer would be "95% percent of the bottles contain between 11.9 and 12.1 ounces". A different way of answering the question would be to estimate the actual distribution of water amounts. The difference here, is that knowing a distribution is often more useful than just knowing the mean, or the variance, or some confidence intervals. Bayesian methods tend to be useful when you want this sort of information which is often the case when you are using it for decision making (or something like game theory). Another uses case is when you are making decisions requiring multiple pieces of information that don't neatly fit together. A simple example is cancer screening. A rational decision about the proper threshold requires you to combine information about (1) The accuracy of your test, (2) The prevalence of the cancer in the population. I will also add that the formula presented in the article is the simple case with discrete distributions. The more general version of the formula can also handle continuous distributions.
- atum47 7y agoI remember the first time I heard of Bayes Theorem , it took me a while to grasp it.
- Nicci00 7y agoI appreciate the lack of math notation, for many with a poor mathematics backgrounds it feels like a huge wall into getting into interesting and useful theories.
- codesushi42 7y agoBayes Theorem hardly requires any math notation at all. It would literally take you less than a minute to understand conditional probability. Yikes.
- bitL 7y agoOh yeah, and the first actually usable form of Bayesian Theorem would be probabilistic graphical models with max-sum algorithm. Good luck mastering that quickly or at all!
- codesushi42 7y agoThat is far from the first usable form of Bayes. I have no idea what point you are making. Bayes Theorem is easily derived algebraically using conditional probability and the chain rule. You can also derive it easily with a Venn diagram. There is barely any notation needed at all here to understand it. If you're struggling with things at that level, it is more likely due to your own laziness, not because the math is hard. Because it is very easy to reason about.
- euroPoor 7y agoChain rule???
- nxpnsv 7y agoI guess some people interested in machine learning doesn’t know about multiplication and division, but i wouldn’t want to depend on their models...
- freehunter 7y agoPossibly true, but just looking at the Wikipedia page for Bayes Theorem, more than half the text on the page is math notation: https://en.wikipedia.org/wiki/Bayes%27_theorem https://en.wikipedia.org/wiki/Bayes%27_theorem It doesn't matter how simple the math actually is, if someone is unfamiliar with mathematical notation it's going to be overwhelming to read.
- allcentury 7y agoI've used Bayes for classification before, is it still the go to for that?
- sacado2 7y agoNaive Bayes is a good baseline since it's both very fast and quite efficient, and it doesn't need a big training set. But it's not often the best model you can find, and it only works if your classes can be linearly separable, i.e it can't model an xor.
- panpanna 7y agoHis alternativ formulation of P(A|B) is really confusing me. Is there a typo in there?
- raibosome 7y agoShameless plug but if anybody wants to look at Naive Bayes classifiers for continuous and categorical dat, I wrote a library here: https://github.com/remykarem/mixed-naive-bayes https://github.com/remykarem/mixed-naive-bayes