14 ms·
Think Bayes: Bayesian Statistics Made Simple (2012)
- partycoder 9y agoThanks for posting this. The Jupyter notebooks (and the fact Github has built-in support for them) really help illustrating the concepts. The book I've used so far to study is "Probability and Statistics: The Science of Uncertainty", by Michael J. Evans and Jeffrey S. Rosenthal. This book is not being published anymore and is free in PDF form.
- bhattisatish 9y agoThe book you mentioned is available at http://www.utstat.toronto.edu/mikevans/jeffrosenthal/ http://www.utstat.toronto.edu/mikevans/jeffrosenthal/
- partycoder 9y agoMay also want to take a look at: "Introduction to Statistics and Probability using R" https://cran.r-project.org/web/packages/IPSUR/vignettes/IPSUR.pdf https://cran.r-project.org/web/packages/IPSUR/vignettes/IPSU... R has builtin functions for most of your needs. You can get a lot done with very few code.
- deleted 9y ago[deleted]
- baxtr 9y agoI regularly forget how Bayes works. Everytime that happens I browse up to that page: https://www.bayestheorem.net/ https://www.bayestheorem.net/ I love the way it’s explained there.
- submeta 9y agoExcellent! Thanks for sharing.
- Gravityloss 9y agoHow Bayes kinda works, or how I see it. Conditional probability (with some caveats that someone in the comments can fill in on): P(a,b) = P(b,a) P(a|b) * P(b) = P(b|a) * P(a) P(a|b) = P(b|a) * P(a) / P(b) a can be model and b can be data so it becomes P(model | data) = P(data | model) * P(model) / P(data) We have or can estimate the things on the right side. We want to ultimately get the thing on the left side.
- deleted 9y ago[deleted]
- pletnes 9y agoMathematically trivial, but great notation to explain the point of Bayes. Brilliant!
- Jach 9y agoTo clear up your first set to have conditional probabilities for everything, Bayes' theorem is just a restatement of the product rule: p(a and b | context c) = p(a|b,c) * p(b|c) = p(b|a,c) * p(a|c) or = p(a|c)*p(b|c) = p(b|c)*p(a|c) if a and b are independent of each other so Bayes only matters when there is dependence: p(a|b,c) = p(a|c) * p(b|a,c) / p(b|c) otherwise it's just p(a|c) = p(a|c) I like to put things in that order because p(a|c) is the "prior belief" and with some handwaving say things like "updated belief = prior belief and new evidence about belief".
- deleted 9y ago[deleted]
- jules 9y agoYou can also think about Bayes' theorem as follows. Suppose we have a logical robot trying to learn about the world. The robot has a collection of hypotheses in its brain. Every time it observes a new fact, it deletes all hypotheses that are incompatible with that fact. For example, suppose it is thinking about the hair colour and eye colour of Joe. It starts with these hypotheses about Joe's (eye colour, hair colour): (eye colour, hair colour) ========================= (blue, blond) (blue, black) (brown, blond) (brown, black) Suppose that it learns that blue eyed people have blond hair. It deletes hypothesis (blue, black) incompatible with it, and keeps only the hypotheses compatible with it: (blue, blond) (brown, blond) (brown, black) Suppose it now learns that Joe has blue eyes. It keeps only the hypothesis compatible with it: (blue, blond) So it has now learned the hair colour. In reality it is not true that all blue eyed people have blond hair. We change the robot's brain and give a weight to each hypothesis indicating how likely it is. Equivalently, we could insert multiple copies of each hypothesis, and the likelihood of a hypothesis is equal to the number of copies of the hypothesis. (blue, blond): 10 (blue, black): 2 (brown, blond): 9 (brown, black): 8 Blue eyed people are more likely to be blond. Those are our hypotheses about the attributes of Joe. Suppose we now learn that Joe has blue eyes. It keeps only the hypotheses compatible with it: (blue, blond): 10 (blue, black): 2 So P(blond hair) = 10/12 and P(black hair) = 2/12. This is all Bayes' theorem is: you have a set of weighted hypotheses, and you delete hypotheses incompatible with the observed evidence. The extra factor in Bayes' theorem is only there to re-normalise the weights so that they sum to 1.
- nafizh 9y agoprevious discussion: https://news.ycombinator.com/item?id=4634843 https://news.ycombinator.com/item?id=4634843
- stablemap 9y agoWith some comments from the author. Hopefully he’ll see this too.
- deleted 9y ago[deleted]
- boostedsignal 9y agoFor those unclear on the concrete (rather than philosophical) difference between Bayesian and frequentist statistics in the first place, I hope it's not inappropriate for me to share this 5-minute example that I wrote a while back: https://news.ycombinator.com/item?id=11096129 https://news.ycombinator.com/item?id=11096129
- partycoder 9y agoAnother problem for the less familiar with the Bayes theorem is what is described as the "Bayesian trap", explained by the youtuber Veritasium: https://www.youtube.com/watch?v=R13BD8qKeTg https://www.youtube.com/watch?v=R13BD8qKeTg
- mycat 9y agoFor me, this book's first chapters explained nicely about ML, MAP and Bayesian using real computer vision problems. The author included helpful visual aids (gaussian plots, contour plots, filters output, etc) http://www.computervisionmodels.com http://www.computervisionmodels.com This is a rather unusual book where it gives primer on probabilistic method that is actually applicable in non computer vision problems. It is Bayesian heavy and rarely touches neural networks; the book is released in 2012, the year deep learning boom started.
- boostedsignal 9y agoYeah, there are lots of good books out there. My goal was to get the point across in a 5-minute read (give or take).
- WhitneyLand 9y agoI hope people agree it’s totally appropriate, and appreciated, thank you for reposting it. This is most of the reason I come here, because people show the good will to share bits of knowledge and experience. Then a whole other benefit, is that when people are willing to do this, their contribution might be critiqued or corrected, which can then sharpen or polish your knowledge and thinking even in areas where you might be very qualified. For some people this would be a nightmare, if they can easily feel angry or hurt when their intellect is challenged, especially when they are an “expert” on the subject. But I suspect most people here feel the opposite. You found a flaw in my results or reasoning? Fucking awesome, you have just make me stronger. edit: I don’t know many other online forums where this dynamic exists, so if anyone does please don’t keep it a secret.
- geekyzen 9y agoSuch a great book. How I got started :)
- epalmer 9y agoMy youngest has Allen Downey as a professor this year. She says he is crazy. And she means this in the best way possible. His productivity is prolific having written Think Java in 13 days. He memorized pictures and bios of all 90 students in the first year class at Olin College of Engineering. Edit typo
- keerthiko 9y agoAllen's classes were always some of the most over-enrolled ever since I can remember at Olin =).
- arcticfox 9y agoMy favorite Allen memory was a modeling contest between him and Mark Somerville. (Physical modeling, of course.) The result was basically a draw, but their approaches were totally different: Allen's was beautifully simple as usual, Mark's brilliantly complex. For anyone interested in Allen's style: http://greenteapress.com/wp/physical-modeling-in-matlab/ http://greenteapress.com/wp/physical-modeling-in-matlab/
- gwern 9y ago> He memorized pictures and bios of all 90 students in the first year class at Olin College of Engineering. Do you know if he was using spaced repetition to do that? I know some teachers have tried that to speed up learning their students.
- sn9 9y agoI would assume so. I can't imagine how you could accomplish that otherwise without having a photographic memory.
- gwern 9y agoTeachers learn their students eventually, by constant regular exposure (which you could consider to be de facto exploiting the spacing effect), so it doesn't require a herculean memory. Spaced repetition software is just a neat trick to speed the process up.
- Nicholas7900 9y agoCOMPOSITE HACKS • If Truly you Are In Need Of A PROFESSIONAL HACKER Who Will Get Your Job Done Efficiently With Swift Response, You Have Met the Right Person. MOTIVE: • I don't care who you are, The reason why I am hired is to simply get the job done however, I engage in"silent" attacks so the target remains unaware of the hack. MY HACKING CAPABILITIES: There are series of people who needs to Hack their target email I.D, social media websites,apps or smartphones, for many reasons such as husband cheating on wife, girlfriend cheating on boyfriend or vice versa. Or important information needed to be dumped/sniffed from the target email address, website, a particular location, or for whatsoever reason you want to hire a Hacker Or As a parent you might need to monitor what your children are doing on social media and on their personal computers so that they don’t get into trouble. Or if you want to hack a private domain email account(business email), or you want to eraze your name from court’s criminal records, perhaps you might want to Hack into the database of any government agency, All these are what I can get done within few hours. Frankly speaking, I always give a 100% guarantee on any job I am been asked to do, because I have always been successful in Almost all my jobs for the past 10years and my clients can testify to that . To hack anything needs time though, but I can provide a swift response to your job depending on how fast and urgent you need. Time also depends on what exactly you want to hack and how serious you are. Enough time with social engineering is required for hacking. So if you want to bind me in a short time, then just don't contact me because I can't hack within one hour,sorry. Basically, time depends on your luck. If its good luck, then it is possible to hack within one hour but, if its in the other way round, it would take few hours. I have seen FAKE HACKERS claiming they can hack in 1hr , but there is no REAL HACKER who can say this (AVOID THEM). If your target account is Facebook, MySpace, Twitter, sexual accounts or other social accounts, Bitcoin wallets, etc, or you want to hack into someones smartphone so you can monitor their calls, read their messages, chats, and social account activities without their notice, Or you wana track someone’s location, Or you want to retrieve lost files,messages, feel free to contact me, we would have a nice deal! I will explain to you how I am going to get the job done. You can also contact me for other Cyber Attacks And Hijackings, i do almost All Any binary options scam victims that is eager to recover lost money can also reach out to me, I'm Always ready to Help. compositehacks@gmail.com
- folksinger 9y agoLet's take a recent election as an example: A Bayesian pollster began with a certain set of prior probabilities. That the college educated were more likely to vote in previous elections, for example, informed the sample population, because it wouldn't make much sense to ask the opinions of those who would stay home. Thus, based on priors that were updated with new empirical data, a new set of probabilities emerged, that gave a certain candidate a high probability of victory. Members of the voting public, aware of this high probability, decided that this meant with certainty that this candidate would win and therefore decided to stay home on election day. In reality the Bayesian models were incorrect as amongst other factors, a much higher number of non-college educated individuals decided to vote and to vote for the other candidate. As it is with Bayesian intelligence, shared as much by pollsters as machine learning algorithms: Real-time heads up display Keeps the danger away But only for the things that already ruined your day.
- clircle 9y agoI suppose you aren't talking about Andrew Gelman... https://www.nytimes.com/interactive/2016/09/20/upshot/the-error-the-polling-world-rarely-talks-about.html https://www.nytimes.com/interactive/2016/09/20/upshot/the-er...
- folksinger 9y agoYou mean the same Andrew Gelman who did not predict the election of Trump and took the time to reflect on the issues with polling methodology? http://andrewgelman.com/2016/12/08/19-things-learned-2016-election/ http://andrewgelman.com/2016/12/08/19-things-learned-2016-el... I'll have to write another poem about pithy rebuttals that cherry-pick a counter-narrative! Now, what rhymes with anecdotal...
- fpoling 9y agoFor me the best so far book on Bayesian probability was "Probability Theory: The Logic of Science: Principles and Elementary Applications" by E. T. Jaynes. The book starts from the deduction of Bayesian theorem from the first principles of logic and shows its applications to a wide range of topics. There is thorough discussion of various "paradoxes" and the author sharply criticizes the frequentist statistics. In addition there is a lot of historical references.
- CalChris 9y agoProbability Theory is available as a PDF. http://www.med.mcgill.ca/epidemiology/hanley/bios601/GaussianModel/JaynesProbabilityTheory.pdf http://www.med.mcgill.ca/epidemiology/hanley/bios601/Gaussia...
- rtehfm 9y agoThanks. I was about to buy the book on Amazon for ~$90 but at least now I can give it a read first to see if it's worth, for me at least, buying.
- vowelless 9y agoGreat book. Unfortunately Jaynes passed away before he could finish it. Still, a great take on probability theory.
- xelxebar 9y agoI came here to recommend this book too! It's a text that definitely allows one to go as deep as they wish very fleshed out references, a good appendix, and lots of comments on directions that can be explored more deeply. It's a book I'm happy to have in dead tree form on my shelf.
- innocentoldguy 9y ago“I broke this rule because I developed some of the code while I was a Visiting Scientist at Google, so I followed the Google style guide, which deviates from PEP 8 in a few places. Once I got used to Google style, I found that I liked it. And at this point, it would be too much trouble to change.” Why would you write a book that targets the Python community and ignore PEP8 styling, inconveniencing an entire community, simply because it would be too much trouble for you to change? “Also on the topic of style, I write “Bayes’s theorem” with an s after the apostrophe, which is preferred in some style guides and deprecated in others.” It is deprecated in all modern style guides and should not be used. You’ll get dinged in college English and writing classes for using this outdated and redundant style. I’m sure this book is great, but, as a point of constructive criticism, I would suggest the author do a better job at adhering to the styles of code and English expected by his target audience, rather than what is comfortable for him.
- bubblesocks 9y agoI'm not sure why you were down-voted. This is a valid point and as a college professor and author, I'm sure Downey would appreciate any feedback that would make his book better.
- bubblesocks 9y agoIt looks like someone is overly sensitive to criticism. Instead of being a coward and voting valid comments down without comment, why don't you add to the discussion by stating why you think the comment was wrong/invalid/etc., thus enlightening the rest of us with your superiority?
- appleiigs 9y agoI can't downvote. The author points out his differences and provides his reasons why. The criticism doesn't add any value to the conversation because it has been addressed already by the author himself.
- 9y ago
- emerged 9y agoMy introduction to Bayesian probability was accidentally reinventing it while trying to invent my own AI system. It naturally followed by constructing a network of information which could be queried to get back whatever had been fed into it and perform deduction/induction.
- platz 9y agoI was surprised to learn about the 'Likelihoodist', an interpretation of bayes that avoids ambiguities with choosing a prior. An Introduction to Likelihoodist, Bayesian, and Frequentist Methods http://gandenberger.org/2014/07/28/intro-to-statistical- http://gandenberger.org/2014/07/28/intro-to-statistical- methods-2/
- raister 9y agoIt should also have one called "Think Markov Chain Monte Carlo" - even the simplest reference is intractable and others began very simplistically and ends incomprehensible enough to disgust the subject altogether.