18 ms·
Statistical Rethinking (2022 Edition)
- spekcular 5y agoThis is a great book. However, I really hate the "Golem of Prague" introduction. It presents an oversimplified caricature of modern frequentist methods, and is therefore rather misleading about the benefits of Bayesian modeling. Moreover, most practicing statisticians don't really view these points of view as incompatible. Compare to the treatment in Gelman et al.'s Bayesian Data Analysis. There are p-values all over the place. Most importantly, this critique fails on basic philosophical grounds. Suppose you give me a statistical problem, and I produce a Bayesian solution that, upon further examination with simulations, gives the wrong answer 90% of time on identical problems. If you think there's something wrong with that, then congratulations, you're a "frequentist," or at least believe there's some important insight about statistics that's not captured by doing everything in a rote Bayesian way. (And if you don't think there's something wrong with that, I'd love to hear why.) Also, this isn't a purely academic thought experiment. There are real examples of Bayesian estimators, for concrete and practical problems such as clustering, that give the wrong estimates for parameters with high probability (even as the sample size grows arbitrarily large).
- kkoncevicius 5y ago> If you think there's something wrong with that, then congratulations, you're a "frequentist". And more than that - if you use bootstrap, or do cross-validation, you are being a frequentist.
- nightski 5y agoHmm. He was comparing the Bayesian models to golems as well, not just frequentist. It was an analogy to all statistical models. Second in the lectures he said that he uses frequentist techniques all the time and that it's often worth looking at it from each perspective. I interpreted it as his problem is not with the methods themselves, but with how they are commonly used in science. To me this made a lot of sense.
- spekcular 5y agoI think I'm misremembering. I read through some of the introductory material in the second edition of his book and found it less critical than I recalled. But in some places, it definitely comes across as hostile (e.g. footnote 107). Also, the sentence "Bayesian probability is a very general approach to probability, and it includes as a special case another important approach, the frequentist approach" is pretty funny. I know the exact technical result he's referring to, but it's clearly wrong to gloss it like that. He does mention consistency once, page 221, but (unconvincingly) handwaves away concerns about it. (Large N regimes exist that aren't N=infinity...)
- nightski 5y agoHonestly I think it is a little hostile. Not towards frequentist directly, but towards the mis-use of frequentist methods in science. He works in ecology and I think he comes across a bunch of crap all the time. He talks at length about the statistical crisis in science and I can't really blame him. But I could see how someone might take this as an attack on the methods themselves.
- kmonad 5y agoI agree. The golem is presented as an analogue to any statistical inference: powerful but ultimately dumb, in the sense that it won't think for you. That's in my opinion the major theme of the book---you have to think and not rely on algorithms/tools/machines...or golems to do that for you.
- rwilson4 5y agoGill's book, Bayesian Methods, is even more dismissive, and even hostile towards Frequentist methods. Whereas I've never seen a frequentist book dismissive of Bayes methods. (Counterexamples welcome!) It boils down to whether you give precedence to the likelihood principle or the strong repeated sampling principle (Bayes prefers the likelihood principle and Frequentist prefers repeated sampling). See Cox and Hinkley's Theoretical Statistics for a full discussion, but basically the likelihood principle states that all conclusions should be based exclusively on the likelihood function; in layman's terms, on the data themselves. This specifically omits what a frequentist would call important contextual metadata, like whether the sample size is random, why the sample size is what it is, etc. The strong repeated sampling principle states that the goodness of a statistical procedure should be evaluated based on performance under hypothetical repetitions. Bayesians often dismiss this as: "what are these hypothetical repetitions? Why should I care?" Well, it depends. If you're predicting the results of an election, it's a special 1 time event. It isn't obvious what a repetition would mean. If you're analyzing an A/B test it's easy to imagine running another test, some other team running the same test, etc. Frequentist statistics values consistency here, more so than Bayesian methods do. That's not to come out in support of one vs the other. You need to understand the strengths and drawbacks of each and decide situationally which to use. (Disclaimer: I consider myself a Frequentist but sometimes use Bayesian methods.)
- spekcular 5y ago"Whereas I've never seen a frequentist book dismissive of Bayes methods. (Counterexamples welcome!)" Indeed! There's a lot of Bayesian propaganda floating around these days. While I enjoy it, I would also love to see some frequentist propaganda (ideally with substantive educational content...).
- kkoncevicius 5y agoA book by Deborah Mayo "Statistical Inference as Severe Testing" might fit.
- spekcular 5y ago
- kgwgk 5y agoSuppose you give me a particle physics problem, and I produce a quantum mechanics solution that, upon further examination, is wrong. If you think there's something wrong with that, then congratulations, you're a "quantum negationist," or at least believe there's some important insight about physics that's not captured by doing everything in a rote quantum way. (The important insight being that GIGO.)
- spekcular 5y agoThe issue isn't that Bayesian methods used incorrectly can have bad frequentist properties. It's that, according to many flavors of Bayesianism, having bad frequentist properties isn't a valid line of critique. You may not believe in the particular stances I'm calling out, but if so, we don't disagree.
- kgwgk 5y agoMaybe we don't disagree. You wrote: > a Bayesian solution that, upon further examination with simulations, gives the wrong answer 90% of time on identical problems If "with simulations" means either "with simulations using a probability distribution different from the prior used in the Bayesian analysis" or "with simulations using a model different from the one used in the Bayesian analysis" are we expected to conclude that there is something wrong with the Bayesian way?
- spekcular 5y agoI mean "with simulations using a probability distribution [for the true parameter] different from the prior used in the Bayesian analysis." (The issue of model error is a separate question.) Yes, in this case would should conclude there is something wrong with the Bayesian way. If you hand me a statistical method to e.g. estimate some parameter that frequently returns answers that are far from the truth, that is a problem. One cannot assume the prior exactly describes reality (or there would be no point in doing inference, because the prior already gives you the truth).
- 5y ago
- agucova 5y agoI think the classes opt for starting with a simple mental model students can adopt, which is gradually replaced with a more robust and nuanced mental model. In this case he wasn't talking just about frequentist methods tho, it's also talking about doing statistics without first doing science (and formulating a causal model). I would be wary of jumping to conclusions from that introduction alone if you haven't seen the rest of the course or the book.
- funklute 5y ago> There are real examples of Bayesian estimators, for concrete and practical problems such as clustering, that give the wrong estimates for parameters with high probability (even as the sample size grows arbitrarily large). Could you give some specific examples, and/or references? This is new to me, and I would like to read deeper into it.
- spekcular 5y agoI was thinking of this paper in particular: https://www.jmlr.org/papers/volume15/miller14a/miller14a.pdf https://www.jmlr.org/papers/volume15/miller14a/miller14a.pdf Some other references: https://www.jstor.org/stable/2241268 https://www.jstor.org/stable/2241268 https://arxiv.org/abs/1308.6306 https://arxiv.org/abs/1308.6306 https://projecteuclid.org/journals/annals-of-mathematical-statistics/volume-36/issue-2/On-the-Asymptotic-Behavior-of-Bayes-Estimates-in-the-Discrete/10.1214/aoms/1177700155.full https://projecteuclid.org/journals/annals-of-mathematical-st...
- mturmon 5y agoThanks for the detail! I took a look at the first paper, the result was new to me. In the vogue days of reversible jump MCMC I played with mixture estimation of the number of components under a basic prior (an approach which gives decent results in Figs 1 and 3), but I never used a Dirichlet process prior for this problem. This paper points out that even this simple approach is problematic because it’s only consistent if the true distribution is such a mixture, and in my case it definitely was not. Anyway, one takeaway, esp. from sec 1.2.1, is that the Dirichlet process prior is not suitable for estimating #components in most cases; it favors small clusters. And indeed, the concept of estimating #components is tricky to begin with, as noted above. Just because you can compute the posterior, doesn’t mean it’s saying what you think it is about the underlying true distribution!
- medstrom 5y agoUh, dude. If you read the book, you'd see the Golem of Prague isn't a parable about frequentist models specifically, it's about all models, period. He calls his Bayesian models golems all the time.
- throwawayarnty 5y agoAgreed. There are situations where frequentist guarantees is what you want rather than optimal estimates of parameters. Frequentists consider the model parameters fixed and allow the data to vary. Bayesian consider the data fixed and allow parameters to vary. For example many software engineering, drug testing, and large scale pipelines want frequentist guarantees, because your system will have varying input data and you want theoretical bounds on what inferences you can make.
- jdreaver 5y agoI've read this book and taken this course twice, and it is easily one of the best learning experiences I've ever had. Statistics is a fascinating subject and Richard helps bring it alive. I had studied lots of classical statistics texts, but didn't quite "get" Bayesian statistics until I took Richard's course. Even if you aren't a data scientist or a statistician (I'm an infrastructure/software engineer, but I've dabbled as the "data person" in different startups), learning basic statistics will open your eyes to how easy it is to misinterpret data. My favorite part of this course, besides helping me understand Bayesian statistics, is the few chapters on causal relationships. I use that knowledge quite often at work and in my day-to-day life when reading the news; instead of crying "correlation is not causation!", you are armed with a more nuanced understanding of confounding variables, post-treatment bias, collider bias, etc. Lastly, don't be turned off by the use of R in this book. R is the programming language of statistics, and is quite easy to learn if you are already a software engineer and know a scripting language. It really is a powerful domain specific language for statistics, if not for the language then for all of the statisticians that have contributed to it.
- jonnycomputer 5y agoI frequently prefer R to python/pandas/numpy for data analysis--even if most of my other programming is in python.
- ivansavz 5y agoHere is a direct link to the playlist: https://www.youtube.com/playlist?list=PLDcUM9US4XdMROZ57-OIRtIK0aOynbgZN https://www.youtube.com/playlist?list=PLDcUM9US4XdMROZ57-OIR... Prof. McElreath has been adding two new videos every week. Also, for anyone who prefers to use the pythons for the coding, I recommend the PyMC3 notebooks https://github.com/pymc-devs/resources/tree/master/Rethinking_2 https://github.com/pymc-devs/resources/tree/master/Rethinkin... There is also a discussion forum related to this repo here https://gitter.im/Statistical-Rethinking-with-Python-and-PyMC3/Lobby https://gitter.im/Statistical-Rethinking-with-Python-and-PyM...
- canyon289 5y agoIm one of the Core devs for Arviz and PyMC! Glad you found those resources useful. If any has any questions feel free to ask them in gitter and we'd be happy to help
- throwoutway 5y agoWill there be other course schedules? These dates don’t work for me unfortunately
- cinntaile 5y agoUnless you're a student there you won't be able to attend the classes and get a grade anyway. You just watch the Youtube video's, he makes new ones each year.
- rossdavidh 5y agoAlso, having self-taught from the previous edition of his excellent book, I can say that it is very useful even if you aren't able to attend his class.
- throwoutway 5y agoOh I misunderstood that.
- elcapitan 5y agoDoes the course require prior statistical knowledge? Couldn't quite figure that out. It looks interesting, and there are python versions of the examples as well..
- jonnycomputer 5y agoI think you'll be fine, actually. I've read through the first edition, and it's kept pretty intuitive.
- agucova 5y agoYou'll probably need some basic notions of statistical distributions and data analysis; I recommend reading the first chapter of the book or the first lecture and seeing whether you're missing anything important.
- qorrect 5y agoI got through it without any priors :).
- elcapitan 5y agoThanks! :)
- dang 5y agoPast related threads: Statistical Rethinking [video] - https://news.ycombinator.com/item?id=29780550 https://news.ycombinator.com/item?id=29780550 - Jan 2022 (10 comments) Statistical Rethinking: A Bayesian Course Using R and Stan - https://news.ycombinator.com/item?id=20102950 https://news.ycombinator.com/item?id=20102950 - June 2019 (14 comments)
- xayfs 5y ago
- laGrenouille 5y ago> The unfortunate truth about data is that nothing much can be done with it This is a fairly strong statement that goes against a lot of other work in data science and information visualization (John Tukey, Edward Tufte, Jacques Bertin, Hadley Wickham, ...). For example, see [0] and [1]. [0] https://en.wikipedia.org/wiki/Exploratory_data_analysis https://en.wikipedia.org/wiki/Exploratory_data_analysis [1] https://courses.csail.mit.edu/18.337/2015/docs/50YearsDataScience.pdf https://courses.csail.mit.edu/18.337/2015/docs/50YearsDataSc...
- nightski 5y agoYou are leaving out a very important part of the sentence - "until we say what caused it". If you listen to the first few lectures you'll understand exactly what he intends with this sentence.
- chiefalchemist 5y agoTo your point...Data has context. It has a source. It likely has flaws and/or (so to speak) bias. To get anything of it It's essential to understand what went into it. Else you'll deceive yourself or your stakeholders and bad decisions will be made.
- hinkley 5y agoThis cleaves very close to an aphorism I stole mercilessly many years ago: charts are for asking questions, not answering them. “What caused it” is the answer, and a graph can reveal just as easily as it can conceal the cause. Lies, damn lies, and statistics.
- laGrenouille 5y agoThanks, though I actually meant to copy the entire thing (my fault). My point was that a lot of people working in data analysis would (strongly) disagree with the idea that we need to model the data in order to do anything with it. Visualisations and tabulations can tell a lot without any mathematical formalism.
- agucova 5y ago
- agucova 5y agoI'm following the course using Julia/Turing.jl and it's simply awesome. Richard McElreath clearly has a talent for teaching, and both the lectures and his book also give a very insightful discussion on the philosophy of science and common pitfalls of common statistical methods. Last semester I took my first classical Probability and Statistics course at my uni, and this course has been positively refreshing in comparison.
- canyon289 5y agoSelf plug: After reading this book if you're looking to continue I recently published a book in the same series with the same publisher. The book is available for open access, though I appreciate folks buying a copy too! https://bayesiancomputationbook.com https://bayesiancomputationbook.com https://www.routledge.com/Bayesian-Modeling-and-Computation-in-Python/Martin-Kumar-Lao/p/book/9780367894368 https://www.routledge.com/Bayesian-Modeling-and-Computation-...
- kfor 5y agoAwesome book, Ravin! I’m waiting for my physical copy to arrive (should be here tomorrow!) before really diving in, but what I’ve skimmed in the digital copy so far is great. Btw I’ve been using PyMC2 since 2010 and contracted a bit with PyMC Labs, so I’m surprised we’ve never bumped into each other!
- canyon289 5y agoThanks for ordering one. Which company are you with? Perhaps we did bump into each other, asking to see if that is the case
- spekcular 5y agoLooks fun! Thanks for sharing. It seems like it covers complementary topics in a very concrete and clear way.
- canyon289 5y agoThanks for checking it out and the feedback. I appreciate it!
- sean_the_geek 5y agoThank you!
- nightski 5y agoJust received my copy, looking forward to tackling it after rethinking!
- canjobear 5y agoI've been reading polemics and tutorials for at least 12 years now arguing for Bayesian methods over Frequentist methods. They all seem persuasive, and everyone seems to be convinced that Bayesianism is the future, and Frequentism was a mistake, and the change to a glorious future where Bayesian methods are the standard way to do stats is just round the corner. It hasn't happened. Meanwhile I've never read a real argument for Frequentism and I don't know where I'd find one, short of going back to Fisher who is not well known for clear writing. What is going on? Is the future of Bayesian statistics just more and more decades of books and articles and code notebooks and great presentations showing how great Bayesianism is? Is it just institutional inertia preventing Bayesian stats from becoming standard, or does Frequentism have a secret strength that keeps it hegemonic?
- agucova 5y agoI guess it really depends on your discipline, but Bayesian methods have become more and more popular in a lot of academic communities, being published alongside papers using frequentist methods, so I wouldn't say it's hegemonic anymore.
- lkozma 5y agoFor those coming from a CS background a possible (crude) intuition sometimes given is that frequentist :: Bayesian ~ worst-case analysis :: average-case analysis There are a good reasons why we don't usually do average-case analysis of algorithms, chief among them that we have no idea how inputs are distributed (another reason is computational difficulty). Worst-case bounds are pessimistic, but they hold.
- zozbot234 5y agoIn CS you're often dealing with adversarial inputs, for which worst-case analysis is obviously the right approach. Not sure that there's anything comparable to that in most statistical settings.
- civilized 5y agoThe biggest problem with Bayesian statistics in practice is the frequent reliance on relatively slow, unreliable methods such as MCMC. The Bayesian methodology community loves to advocate for packages like Stan, claiming that they make Bayesian stats easy. This is true... relative to Bayesian stats without Stan. But these packages are often much, much harder to get useful results from than the methods of classical statistics. You have to worry about all sorts of technical issues specific to these methodologies, because these issues don't have general technical solutions. Knowing when you even have a solution is often a huge pain with sampling techniques like MCMC. So you have to become an expert in this zoo of Bayesian technicalities above and beyond whatever actual problem you are trying to solve. And in the end, the results usually carry no more insight or reliability than you would have gotten from a simpler method. I recommend learning about Bayesianism at a philosophical level. Every scientist should know how to evaluate their results from a Bayesian perspective, at least qualitatively. But don't get too into Bayesian methodology beyond simple methods like conjugate prior updating... unless you are lucky enough to have a problem that is amenable to a reliable, practical Bayesian solution.
- rabaath 5y agoThis book is amazing. In my opinion the best book to get started with advanced statistics (all statistics, not just Bayesian statistics).
- huijzer 5y agoOne problem though. If you start with McElreath, you will likely find all books which require you to wrangle your brain into sided p-values and confidence intervals stupid
- akdor1154 5y agoI wanna buy the hardcopy textbook but still have access to an epub version - do any retailers allow this? Linked publisher site doesn't seem to.
- reactspa 5y agoThere's so much great content on the internet that's not easily discoverable. This is one. If you have discovered a great resource for intuitively learning about fat-tailed-distributions related mathematics, please share. I have fallen into the Taleb rabbit-hole and would really like to gain an *intuitive* understanding of what he's talking about when he mentions topics such as gamma distributions, lognormal distributions, loglikelihood.
- deleted 5y ago[deleted]
- throwawaybayyes 5y agoFrom slide 5 of lecture 2: Explanations with more ways to produce the data are more plausible. Sorry, I disgress, the bias/variance trade off implies that you should ponder the complexity of the model to avoid overfitting. I don't know much about Bayesian data analysis, but I think the first idea to be introduced is not the concept of likelihood, Bayes theorem is the foundation and I think it should be introduced before likelihood. On slide 31, it is not clear if you state a causal model before making observations, you should indicate a prior before making the observation so that you can then see how it updates, if done otherwise you are using the observations to elaborate the model and then the analysis is crookely designed. On slide 35, the concept of relative plausibility seems to me a little sloopy to develop some theory. On slide 48: In the context of confidence intervals: "95% is obvious superstition". I think 95% interval is using a standard way to communicate results that gives a useful information and avoid the question of why you computed, for example, the 93.728 confidence interval.
- james-redwood 5y agoNot only is this a great book, but something that hasn’t been mentioned is that it’s also fantastic for absolute statistical beginners. My reasonably intelligent 12 year old nephew had no issues understanding much of it.