3 ms·
Bayesian Data Analysis, Third edition (2013) [pdf]
- moscoe 1y agoRelated course materials here: https://sites.stat.columbia.edu/gelman/book/ https://sites.stat.columbia.edu/gelman/book/
- mcdonje 1y agoI'm a fan of the stats blog hosted by Columbia that Gelman is the main contributor to: https://statmodeling.stat.columbia.edu https://statmodeling.stat.columbia.edu
- dpflan 1y agoThanks for sharing, any particular articles that had last impact on you?
- mcdonje 1y agoidk about impact, but here are a couple I liked: - https://statmodeling.stat.columbia.edu/2025/08/25/what-writing-a-failed-rock-paper-scissors-program-taught-me-or-should-have-taught-me-about-sample-size-and-uncertainty/ https://statmodeling.stat.columbia.edu/2025/08/25/what-writi... - https://statmodeling.stat.columbia.edu/2025/09/04/assembling-an-unbiased-jury/ https://statmodeling.stat.columbia.edu/2025/09/04/assembling...
- cubefox 1y agoHere are the articles that were popular on HN over the years: https://hn.algolia.com/?q=statmodeling.stat.columbia.edu https://hn.algolia.com/?q=statmodeling.stat.columbia.edu
- 1u15 1y agoBeyond “power pose”: Using replication failures and a better understanding of data collection and analysis to do better science https://statmodeling.stat.columbia.edu/2017/10/18/beyond-power-pose-using-replication-failures-better-understanding-data-collection-analysis-better-science/ https://statmodeling.stat.columbia.edu/2017/10/18/beyond-pow... You need 16 times the sample size to estimate an interaction than to estimate a main effect https://statmodeling.stat.columbia.edu/2018/03/15/need16/ https://statmodeling.stat.columbia.edu/2018/03/15/need16/ Debate over effect of reduced prosecutions on urban homicides; also larger questions about synthetic control methods in causal inference. https://statmodeling.stat.columbia.edu/2023/10/12/debate-over-effect-of-reduced-prosecutions-on-urban-homicides-also-larger-questions-about-synthetic-control-methods-in-causal-inference/ https://statmodeling.stat.columbia.edu/2023/10/12/debate-ove... Bayesians moving from defense to offense: “I really think it’s kind of irresponsible now not to use the information from all those thousands of medical trials that came before. Is that very radical?” https://statmodeling.stat.columbia.edu/2023/12/23/bayesians-moving-from-defense-to-offense-i-really-think-its-kind-of-irresponsible-now-not-to-use-the-information-from-all-those-thousands-of-medical-trials-that-came-before-is-that-very/ https://statmodeling.stat.columbia.edu/2023/12/23/bayesians-...
- asdev 1y agoLooking for more self study statistics resources for someone with a CS degree, any other recs?
- fishmicrowaver 1y agoProbability Theory by Jaynes if you'd like more bayes
- 3eb7988a1663 1y agoI think Statistical Rethinking [0] is a far more approachable first entry. The author posts his video lectures on Youtube which are excellent and should be watched with the book. The book gets way less into the mathematical weeds than other texts, so a working statistician would require something deeper. [0] https://en.wikipedia.org/wiki/Statistical_Rethinking https://en.wikipedia.org/wiki/Statistical_Rethinking
- sebg 1y ago2024 videos / lectures on github here -> https://github.com/rmcelreath/stat_rethinking_2024 https://github.com/rmcelreath/stat_rethinking_2024
- CuriouslyC 1y agoStart here: https://www.inference.org.uk/itprnn/book.pdf https://www.inference.org.uk/itprnn/book.pdf It's a little dated now but it connects Bayesian statistics with neural nets and information theory in an elegant way.
- mamonster 1y agoStart with statistics by David Freedman. It is very approachable as an introduction, not too theory heavy, can get a handle on all of the "main" issues. Afterwards, you have 2 options: 1) Do you want "theoretical" knowledge(math background required)? If so, then you need to get a decent mathematical statistics book like Casella-Berger. I think a good US CS degree grad could handle it, but you might need to go a bit slow and google around/ maybe fill in some gaps in probability/calculus. 2)Introduction to Statistical Learning is unironically a great intro to "applied" stats. You have most of the "vanilla" models/algorithms, theoretical background behind each but not too much, you can follow along with the R version and see how stuff actually works and exercises that vary in difficulty. With regards to Gelman and Bayesian data analysis, I should note that in my experience the Bayesian approach is 1st year MS /4th year of a Bachelors in the US. It's very useful to know and have in your toolbox but IMO it should be left aside until you are confident in the "frequentist" basics.
- atdt 1y agoI am interested in this topic, but this textbook is too daunting for me. What I'd love is a crash course on Bayesian methods for the working systems performance engineer. If you, dear reader, happen to be familiar with both domains: what would you include in such a course, and can you recommend any existing resources for self-study?
- esafak 1y agohttps://github.com/CamDavidsonPilon/Probabilistic-Programming-and-Bayesian-Methods-for-Hackers https://github.com/CamDavidsonPilon/Probabilistic-Programmin... https://www.oreilly.com/library/view/bayesian-methods-for/9780133902914/ https://www.oreilly.com/library/view/bayesian-methods-for/97...
- JHonaker 1y agoMy go to for teaching statistics is Statistical Rethinking. It’s basically a course in how to actually thing about modeling: what you’re really looking for is analyzing a hypothesis, and a model may be consistent with a number of hypotheses, figuring out what hypotheses any given model implies is the hard/fun part, and this book teaches you that. The only drawback is that it’s not free. (Although there are excellent lectures by the author available for free on YouTube. These are worth watching even if you don’t get the book.) I also recommend Gelman’s (one of the authors of the linked book) Regression and Other Stories as a more approachable text for this content. Think Bayes and Bayesian Methods for Hackers are introductory books from a beginner coming from a programming background. If you want something more from the ML world that heavily emphasizes the benefits of probabilistic (Bayesian) methods, I highly recommend Kevin Murphy’s Probabilistic Machine Learning. I have only read the first edition before he split it into two volumes and expanded it, but I’ve only heard good things about the new volumes too.
- huijzer 1y agoYep 100% came here to say the same. Helped me a lot during the PhD to get a better understanding of statistics.
- kianN 1y agoThis is my favorite book on statistics. Full stop. The author Andrew Gelman created a whole new branch of Bayesian statistics with both his theoretical work on hierarchical modeling while also publishing Stan to enable practical applications of hierarchical models. It took me about a year to work through this book on the side (including the exercises) and it provided the foundation for years of fruitful research into hierarchical Bayesian models. It’s a definitely not an introductory read, but for any looking to advance their statistical toolkit, I cannot recommend this book highly enough. As a starting point, I’d strongly suggest the first 5 chapters for an excellent introduction to Gelman’s modeling philosophy, and then jumping around the table of contents to any topics that look interesting.
- SilverElfin 1y agoIs there a good book that covers statistics as it is applied to testing - like for medical research or as optimization or manufacturing or whatever?
- kianN 1y agoThis book is very relevant to those fields. There is a common choice in statistics to either stratify or aggregate your dataset. There is an example in his book discussing efficacy trials across seven hospitals. If you stratify the data, you lose a lot of confidence, if you aggregate the data, you end up just modeling the difference between hospitals. Hierarchical modeling allows you to split your dataset under a single unified model. This is really powerful for extracting signal for noise because you can split your dataset according to potential confounding variables eg the hospital from which the data was collected. I am writing this on my phone so apologies for the lack of links, but in short the approach in this book is extremely relevant of medical testing.
- greymalik 1y agoIt’s unclear which post you’re referring to - can you clarify which book you mean by “this book”?
- 1y ago
- tomhow 1y agoPreviously: Bayesian Data Analysis, Third Edition [pdf] - https://news.ycombinator.com/item?id=23091359 https://news.ycombinator.com/item?id=23091359 - May 2020 (48 comments)
- g9yuayon 1y agoI can attest how useful Bayesian analysis is. My team recently needed to sample from many millions of items to test their qualities. The question is that given a certain budget and expectation, what's the minimum or maximum number of items that we need to sample. There was an elegant solution to this problem. What was surprising, though, was how reluctant the engineers are to learn such basic techniques. It's not like the math was hard. They all went through the first-year college math and I'm sure they did reasonably well.
- some_guy_nobel 1y agoWhat were they reluctant to learn? Why do they need to learn it? Plenty of engineers have to take an introductory stats course, but it's not clear why you'd want your engineers to learn bayesian statistics? I would be surprised if they could correctly interpret a p-value or regression coefficient, let alone one with interaction effects. (It'd be wholly useless if they could, fwiw). It'd be nice if the statisticians/'data scientists' on my team learned their way around the CI/CD pipelines, understood kubernetes pods, and could write their own distributed training versions of their pytorch models, but division-of-labor is a thing for a reason, and I don't expect them to nor need them to.
- g9yuayon 1y agoI guess I have a different philosophy: whoever owns the problem should learn everything necessary to solve the problem. In my case, the engineers showed no interests in learning the algorithm and the math behind it. For instance, when they built the dashboard for the testing, they omitted a few important columns and got the column names wrong. When I tested them on their understanding of the method, there was none. To say the least, my team should know enough to challenge me in case I made any mistake, or so I assume. On a side note, I believe it is an individual's responsibility to find the coolness in their project. What's the fun of building a dashboard that I have done a thousand times? What's the fun of carrying out a routine that does not challenge me? But solving a problem in a most rigorous and generalized way? That is something in which an engineer can find some fun. Or maybe it's just me.
- j7ake 1y agoIs Bayesian data analysis relevant anymore in the era of foundation models and big data?
- canjobear 1y agoWhy would it not be? You can use big data and neural nets to fit Bayesian models (variational inference).
- j7ake 1y agoI meant specifically the book, which doe not have any of those things you mentioned. Also nobody fits neural networks and use variation inference using any priors that aren’t some standard form that makes algorithm easy
- tech_ken 1y agoYeah definitely! People still need to do statistical inference in 2025 (see ex. the field of econometrics).
- mitthrowaway2 1y agoEven in this era, there are some problems for which data is extremely limited. Those IMO tend to be the problems in which Bayesian techniques shine the most.
- CuriouslyC 1y agoYes, because Bayes' rule is fundamental if you're reasoning probabalistically. Bayesian methods produce better results with quantified uncertainty, we just don't have efficient methods to compute them for deep models yet.
- memming 1y agoFoundation models can be seen as approximate amortized posterior inference machines where the posterior is conditioning on the pre-training data. However, the uncertainty is usually ignored, and there may be ways to improve the state of the art if we were better Bayesians.
- canyon289 1y agoBDA is THE book to learn Bayesian Modeling in depth rigorously. For different approaches there are a number shared here like Statistical Rethinking from Richard McElreath or Regression and other stories which Gelman and Aki wrote as well. I also write a book on the topic which is focused a code and example approach. It's available for open access here. https://bayesiancomputationbook.com https://bayesiancomputationbook.com
- jakubmazanec 1y agoFor beginners, I found Doing Bayesian Data Analysis by John Kruschke to be much better. Easier to read and comprehend.
- pks016 1y agoWaiting for the Bayesian workflow book!
- some_guy_nobel 1y agoI first found Gelman from this wonderful visual explanation of Linear Regression: https://mlu-explain.github.io/linear-regression/ https://mlu-explain.github.io/linear-regression/ It cited Regression and Other Stories (though not the Bayesian chapters, which I'm now inspired to dig into before checking this out).
- piqufoh 1y agoWhile we're here - I've gained a lot from "Data Analysis: A Bayesian Tutorial" by DS Sivia and J Skilling. It's a graduate level text, and I found the chapters very concise and the subject well-laid out. It was one of those books that gave me continuous insight and fresh inspiration - even though it's more than 10 years old.
- kalx 1y agoThis helped me so much during my phd
- mkw5053 1y agoBefore my 2 year old, I led a math book club in SF. This was one of the books I taught/led with the group and it's still one of my favorites.