4 ms·
Bayesian Analysis with Python
- kingkongjaffa 3y ago$50 (price on amazon) is a lot for an amateur self published author book. Thats tenured professor textbook money. Pakt publishing books can be written by anyone with even a surface level understanding of a topic.
- kingkongjaffa 3y agoHeres a review from one of the other books by pakt and this author: If you are an inexperienced programmer or new to Python/Jupyter/Anaconda DO NOT BUY THIS OR ANY OTHER PACKT Publishing book as the code contains errors that are difficult to rectify. Packt Publishing DOES NOT verify code like the CRC Press - for instance, Statistical Rethinking by CRC press - i.e. Bayesian Analysis in R - has code AND excellent content that is helpful for both the academic and practitioner. Also, the Packt Publishing description/outline of probability is weak, at best, and is confusing to many of my students. If correcting Python code is not a big deal for you then Packt books are a nice intro. But why buy something that you have to fix before you can start working with it? This is a December 2018 update - DO NOT BUY THIS OR ANY OTHER PACKT PUBLISHING BOOK UNLESS YOU CAN VERIFY THERE IS AN ERRATA FILE TO ACCOMPANY IT. PACKT PUBLISHING DOES NOT PRODUCE RELIABLE TEXTS. MY STUDENTS HAVE HAD A TERRIBLE TIME WITH THIS BOOK – THE COMPANY HAS NOT RESPONDED TO ANY QUESTIONS/REQUESTS - THE UNIVERSITY WHERE I TEACH IS NOW LOOKING CLOSELY AT THE VALIDITY OF THESE PUBLICATIONS.
- ayhanfuat 3y agoIt is not just code. Content in general is horrible, too. You put together a few badly written blog posts and it becomes a Packt published book.
- _yb2s 3y agoI want to second the recommendation for Statistical Rethinking from that review, it’s one of the best books ever written on computational Bayesian inference. However, it is R based, and therefore arguably not an alternative to the book we are discussing here.
- HuShifang 3y agoAs it happens, there's a PyMC implementation of the 1st and 2nd editions of Statistical Rethinking here: https://github.com/pymc-devs/pymc-resources https://github.com/pymc-devs/pymc-resources (I think the author of the book discussed above, Osvaldo Martin, is the primary or sole contributor for the Rethinking implementations, in fact -- he had a full implementation in his own repo (https://github.com/aloctavodia/Statistical-Rethinking-with-Python-and-PyMC3 https://github.com/aloctavodia/Statistical-Rethinking-with-P...) before deprecating it in favor of the above-linked one.)
- perrygeo 3y agoAgreed - that book is phenomenal at communicating the theory and mindset of Bayesian analysis. My favorite quote from the lectures - "Statistics should be subbordinate to Science." A very healthy emphasis on logic and proper model design - there will be no shoving data into black boxes! R is a good language for this but he uses a library of convenience functions that are not on CRAN and are effectively just for educational purposes. So even if you want to stick with R, you'll need to translate your code into production-ready libraries anyway. There are several nearly-complete translations based on other R packages as well as ported to Julia and Python.
- disgruntledphd2 3y agoOr you could just put his code into production (source: I worked at a super successful gaming startup that had this code scattered all over their repo). They had a bunch of other code problems too, but that was definitely the weirdest thing I saw in my (very short) stay there.
- perrygeo 3y agoI thought the whole paragraph about "This package is never going to be on CRAN" was a warning label :-) I agree, it works beautifully and performance is ok for desktop use.
- fbdab103 3y agoYou are cheating yourself if you only read the book. His lecture series covering the material on Youtube is phenomenal. I went through the book + season 1 videos, and had a glance at some of the season two videos. The season two has some visuals that made some intuition click for me. Now I see he has a 2023 playlist, which may yield even further improvements: https://www.youtube.com/playlist?list=PLDcUM9US4XdPz-KxHM4XHt7uUVGWWVSus https://www.youtube.com/playlist?list=PLDcUM9US4XdPz-KxHM4XH...
- apwheele 3y agoAgree about Pakt, but not about $50 self publish per se (nor the tenured prof remark). I am in the process of writing a code book, but it is compiled via quarto so at least you know the code works as expected. This is my favorite helpful advice for pricing, https://pubsonline.informs.org/doi/abs/10.1287/mnsc.2020.3602 https://pubsonline.informs.org/doi/abs/10.1287/mnsc.2020.360.... Estimate max someone will buy, then divide by two (assuming no marginal cost, like e-book).
- HuShifang 3y agoFWIW here's Aki Vehtari's approving share on his (and Gelman et al's) blog. https://statmodeling.stat.columbia.edu/2024/02/08/bayesian-analysis-with-python/ https://statmodeling.stat.columbia.edu/2024/02/08/bayesian-a... The author was also a lead author on a CRC red-series book, Bayesian Modeling and Computation in Python Learning, published a few years ago (short review here): https://academic.oup.com/jrsssa/article/185/Supplement_2/S764/7069537?login=false https://academic.oup.com/jrsssa/article/185/Supplement_2/S76...
- jononor 3y agoI think that price point is fine. Self publishing has a lot of variance of course, but I would not categorically say that 50 dollars is too high. We as potential readers just need to do our due diligence, checking reviews etc. A book not worth 50 USD is not worth 30 or even 20 in my opinion - it is the wanted time spent reading that is the main issue.
- tussa 3y agoI'd never buy anything from Pakt without seeing it endorsed by someone I trust. It's a shame because there is good stuff in their catalogue, but it's littered with trash.
- sheepscreek 3y agoWhat’s your opinion of Manning compared to Pakt? Both actively pursue potential authors. I know this because I’ve been contacted by both. Made me realize I have neither the will nor the patience to write a book - being an author sounds great on paper (pun intended).
- abecedarius 3y agoManning seems fine to me. I've read a few books from them, from quite good to blah. I've read one book from Packt that was quite good; they've also published literal plagiarism (which they withdrew after the original author, a friend, tweeted about it -- this wouldn't have come to my attention otherwise). Can't recall seeing another that interested me, flipping through the pages.
- tussa 3y agoI'd say black and white. I haven't seen anything as rubbish from Manning.
- deleted 3y ago[deleted]
- bradyat 3y agoThink Bayes by Allen Downey is an amazing and free course teaching Bayesian statistics in Python. Save yourself the $50 and try it first: https://allendowney.github.io/ThinkBayes2/ https://allendowney.github.io/ThinkBayes2/
- mmasu 3y agoThanks for posting this as it seems very interesting. Did you go through the whole book/course? the preface claims that “You don’t need to know calculus or linear algebra. You don’t need any prior knowledge of statistics”. I am always a bit skeptical when i read these claims as the texts tend ultimately to be a bit superficial; it would be nice to have an informed opinion about it before starting it out :-) thanks in advance.
- bradyat 3y agoYeah I have completed it, and I really like his teaching style. I’d say it might be a little tough if you have absolutely no knowledge of statistics. But as someone who wanted to do a deeper dive into Bayesian stats specifically I found it a good resource. The reason I ended up doing it was precisely because I felt that the section on bayes in most stats courses is too superficial
- mmasu 3y agothanks!
- compumetrika 3y agoI worked through large chunks of the book a long time ago (perhaps an earlier version). I agree with the other responder-- if you've had zero stats it may be more difficult, but the author does an excellent job of discretizing almost everything, which means that integrals and derivatives are replaced with adding and subtracting. For this book I think the most advanced math you need is multiplication and division. It's a clever trick that relies on the fact that computers are really good at multiplying 10,000 things by 10,000 other things and adding them up (which is what discretization implies here). Before computers that was much harder to do so you needed clever math tricks to "do it all at once," which gets you to all the analytical methods -- calculus, etc. Still all very useful! Just hard to teach to a broad audience. Downey really leans into the technical advantage to computers provide and thus can cover a lot of material before getting into complicated math.
- richrichie 3y agoI would recommend BDA3 and Stan. http://www.stat.columbia.edu/~gelman/book/ http://www.stat.columbia.edu/~gelman/book/
- argiopetech 3y agoBDA3 is undoubtedly the authoritative source, but it's tough going without prior experience.
- fbdab103 3y agoI will second that perspective. Would steer clear as an introductory text.
- richrichie 3y agoIt is a challenging book. But it is an introductory level material on Bayesian statistics. One can teach an undergrad course from the book. IMO, doing computation along the way makes the process enjoyable. Stan / julia-turing / pymcmc all good tools.
- somethingsome 3y agoSelf plug for stan on windows in a docker dev environment: https://github.com/dbonattoj/cookiecutter-pystan https://github.com/dbonattoj/cookiecutter-pystan
- sito42 3y agonumpyro is an underrated library that runs on jax which makes it easy to put on the gpu. They have a nice suite of examples. https://github.com/pyro-ppl/numpyro https://github.com/pyro-ppl/numpyro
- canyon289 3y agoNumpyro is fantastic, great syntax and great contributors as well. PyMC now supports jax and numba as well, and can use the numpyro samplers. https://www.pymc.io/projects/examples/en/latest/samplers/fast_sampling_with_jax_and_numba.html https://www.pymc.io/projects/examples/en/latest/samplers/fas... Between all these tools we hope the modern bayesian can find the one that works best for them
- mvanaltvorst 3y agoIn my experience, PyMC leads to models that are orders of magnitudes slower than equivalent models written in JAGS. Profiling is also extremely tedious, and there is no section in the PyMC docs that touches upon model performance. I really like PyMC's API, but as soon as you move towards bigger datasets JAGS or Stan seem to be the only practical options.
- canyon289 3y agoPyMC has a pretty active community where we help people with questions like these. Its hard to come up with one doc as folks have different hardware, datasets, models etc. I hope though you'll the devs and community to be friendly though! https://discourse.pymc.io/t/how-to-increase-sampling-speed-with-pm-sample/11108/3 https://discourse.pymc.io/t/how-to-increase-sampling-speed-w...
- ckrapu 3y agoFor models with >100 parameters, there are theoretical reasons for why JAGS can fail badly. It has to do with the mixing time of Gibbs samplers versus Hamiltonian Monte Carlo.
- hackandthink 3y ago"Information Theory, Inference, and Learning Algorithms" This is still my favorite statistics book. MacKay is a staunch Bayesian, but his writing is relaxed and witty. http://www.inference.org.uk/itila/book.html http://www.inference.org.uk/itila/book.html
- jononor 3y agoI recently found the Statistical Rethinking course on YouTube. I found it super refreshing in its very applied focus, direct advice and thorough pedagogy. It assumed that one has a little bit of stats and Bayesian thinking from before, but I found it accessible with just basic introduction to stats in engineering and ML. https://youtube.com/playlist?list=PLDcUM9US4XdPz-KxHM4XHt7uUVGWWVSus&si=eWys_Us1e57TLPg5 https://youtube.com/playlist?list=PLDcUM9US4XdPz-KxHM4XHt7uU...
- malshe 3y agoRelatedly, I think for introductory Bayesian analysis, Statistical Rethinking is the best book. It uses R code but you will find GitHub repositories for translated Python code. In the R ecosystem, brms is currently the best (in my opinion) package for people who want to learn Bayesian analysis. Here is an online free book that translated all the original R code to brms code: https://bookdown.org/content/4857/ https://bookdown.org/content/4857/
- nextos 3y agoI prefer Gelman, Hill & Vehtari Regression and Other Stories (ROS). It's also free: https://avehtari.github.io/ROS-Examples https://avehtari.github.io/ROS-Examples. It's a great prequel to Gelman et al. Bayesian Data Analysis (BDA). I have a formal background in Bayesian inference, I do statistics for a living and I still learned many things from ROS. Yet, it is totally approachable for beginners. Statistical Rethinking is something you can read after ROS, before BDA. Personally, I think it is a great book, but it's not a good first book and I think it would benefit from some editing. For example, chapter titles are funny, but totally uninformative if you are browsing the book. Nonetheless, I still think it's a great read.
- blindstitch 3y agoI still refer to the R notebooks I created in doing the question sets for RaOS. A very high quality book.
- malshe 3y agoOh yes, it is an excellent book too.
- canyon289 3y agoDisclaimer, I wrote a Bayesian Book with Osvaldo published by CRC. You can read it here for free! https://bayesiancomputationbook.com/welcome.html https://bayesiancomputationbook.com/welcome.html While I'm mostly happy with this book. Yet there's also many improvements we can make. We're compiling suggestions for how to make a second edition better so if you have any please share! We also recently updated the code to the newest version of PyMC. I also use(d) Bayesian stats extensively in practice at SpaceX, sweetgreen, now Google. It's powerful stuff, it was absolutely crucial in many analyses I did. It's my work at SpaceX that really made me appreciate Bayesian statistics. To respond to some comments below. * Osvaldo is a tenured professor, not an amateur. * PyMC can compiles to Jax and rust backends now speeding things up. Stan, Numpyro and other PPLs are fantastic too pick what works for you * There are many ways to learn Bayesian stats these days. Statistical Rethinking and BDA3 are great. We felt like there was a gap which is why we wrote another book. CRC agreed which is why they published it. Pick what works best for you. I did not write this Packt book with Osvaldo but I do think the material itself is great and it fits a certain audience. If you have questions about learning or using Bayesian stats, I'm happy to answer them!
- harperlee 3y agoThanks for your comment! Typically people have chosen one book, enjoyed it to some extent, and praise it, but are not well posed to comment on comparative advantages of each book; at most they have skimmed the rest. You on the other hand have made a sizable time and effort (and possibly money?) investment on writing a book, and you stated that you made that decision after identifying the gap, so I assume that you have a very grounded opinion on the differences / points of view / strengths and weaknesses of each of those books? It would be great if you could expand a little bit more about each of them.
- canyon289 3y agoThank you for asking. I find this stuff super interesting. I write about it on my blog https://ravinkumar.com/ https://ravinkumar.com/ and here's a video for the most common question I get about bayes vs frequentist https://www.youtube.com/watch?v=foSPfzYs4yY https://www.youtube.com/watch?v=foSPfzYs4yY For recommendations here's my suggestions split by persona * I want a PHD in statistics or to write novel research - Go read BDA3 cover to cover. Gelman etal are amazing, and amazingly brilliant. The book is dense though, even after years of experience I need to now read chapters 2 or 3 times and write out proofs * I want have a smooth on ramp into Bayesian stats with lots of code and beautiful writing. I also like video lectures and undergraduate college courses - Statistical Rethinking. As noted below Richard does a wonderful job explaining these concepts with metaphors like golems, interspersing it with this experience as an anthropologist, and using his teaching experience to write well structured introduction into Bayesian stats. This book does assume you understand basic statistics and probability theory. * I want a great comparison of Bayesian vs Frequentist stats- This book covers both topics well, compares them fairly, and has all the proofs to back things up https://www.routledge.com/Understanding-Advanced-Statistical-Methods/Westfall-Henning/p/book/9781466512108 https://www.routledge.com/Understanding-Advanced-Statistical... * I'm a programmer type person that likes hands on "build from scratch" using code - Allen Downey's think bayes builds up bayes theorem from Numpy arrays. He's also a brilliant instructors * I want to read about the history and people and politics - Bernoulli's fallacy, The Theory That Would Not Die:, and Probably Overthinking it are all "non mathy" great armchair readings https://cup.columbia.edu/book/bernoullis-fallacy/9780231199957 https://cup.columbia.edu/book/bernoullis-fallacy/97802311999... https://www.amazon.com/Theory-That-Would-Not-Die/dp/0300188226 https://www.amazon.com/Theory-That-Would-Not-Die/dp/03001882... https://www.allendowney.com/blog/ https://www.allendowney.com/blog/ * Im an applied practitioner that is focusing more on my specific problem and I need to use the latest PPLs and code to get it done - This is my book. I had to make estimations in SpaceX supply chain with some quick deadlines and I didn't have time to take an undergrad course. I also needed my code to be robust, testable, and scalable. I didn't find that other books provided this so that's why I wrote this with CRC and Osvaldo. Osvaldo and I are heavy contributors to PyMC, ArviZ, Preliz and other libraries so naturally we take a code first approach. https://bayesiancomputationbook.com/welcome.html https://bayesiancomputationbook.com/welcome.html * (Shameless self promotion) Im a professional, I need to learn fast, and my company will pay for training - For this specific niche me and other Bayesian colleagues created an online course designed specifically for professionals. Yes it's expensive so let me plainly state no one at any point needs to spend any money to learn Bayesian stats. That being said hundreds of people have purchased this course and the feedback we've gotten on this course has been quite positive. So I want to underscore before Hacker news rips me apart. No one is being forced to buy this, if you want this style course here it is, if you don't there's many many ways to learn Bayesian stats. https://www.intuitivebayes.com/ https://www.intuitivebayes.com/ You can sign up for the Gaussian Process course for free if you'd like https://www.intuitivebayes.com/gaussian-processes https://www.intuitivebayes.com/gaussian-processes As you can tell I'm very fascinated by generative modeling. I find it fun to think about. I also frankly find it lucrative. It's helped me job hop across some pretty cool companies and get through the Google interview. The combination of strong programming skillset with applied generative mathematics is only heating up so I feel lucky to have chanced upon it, and also thankful that many others before me put great code and reading material out there so I could learn myself.