5 ms·

Stan is best viewed in my mind as a successor to BUGS (Bayesian Inference Using Gibbs Sampling) which more people may have heard of. In fact, there are lots of

Stan is best viewed in my mind as a successor to BUGS (Bayesian Inference Using Gibbs Sampling) which more people may have heard of. In fact, there are lots of players in the probabilistic programming space now, personally I like the model of "Infer.NET" [1] from Microsoft Research, as I find variational and approximate variational inference a good solution to my problems and I like coding models in nearly C# more than in Stan format. If you want to get started as fast as possible with computational Bayesian inference and don't need the performance and advanced features Stan, I'd recommend emcee [2] which is a lightweight python MCMC sampler. Final recommendation to the HN crowd: Bayesian Data Analysis 3rd Edition [3] should be on your desk if you are thinking about using any of this software. [1] http://research.microsoft.com/en-us/um/cambridge/projects/infernet/ http://research.microsoft.com/en-us/um/cambridge/projects/in...] [2] http://dan.iel.fm/emcee/current/ http://dan.iel.fm/emcee/current/ [3] https://www.crcpress.com/Bayesian-Data-Analysis-Third-Edition/Gelman-Carlin-Stern-Dunson-Vehtari-Rubin/9781439840955 https://www.crcpress.com/Bayesian-Data-Analysis-Third-Editio...

tel 11y ago
+1 on all of this (except I'm a fan of full MCMC, have you used ADVI?)
- probdist 11y ago
  Haven't tried ADVI yet, but I think it is (likely) a great idea. In general automatic differentiation is a wildly powerful technique and is underutilized in computational science. Even in pure optimization problems people rarely leverage it to get J or H information as often as they probably should. I'll admit to not being super familiar with all the machinery under the hood in the ADVI paper [1]. It wasn't out when I started working on my current project. [1] http://arxiv.org/abs/1506.03431 http://arxiv.org/abs/1506.03431
  - proditus 11y ago
    do let me know how it goes if you do try ADVI. we're also working on making it more robust to initialization and step-sizes. stay tuned.
    - mjw 11y ago
      AVDI is incredibly awesome. Thanks so much for this work. Really nice readable paper on it too. As someone working with large datasets I think automating variational inference + SGD to work with a broad class of models is really the way forwards and a big force multiplier for machine learning.
kristjankalm 11y ago
some useful books available as a PDF: David Barber "Bayesian Reasoning and Machine Learning" http://web4.cs.ucl.ac.uk/staff/D.Barber/pmwiki/pmwiki.php?n=Brml.HomePage http://web4.cs.ucl.ac.uk/staff/D.Barber/pmwiki/pmwiki.php?n=... -- Thorough and systematic, accompanied by a Matlab library Marc Steyvers "Computational Statistics with Matlab" http://psiexp.ss.uci.edu/research/teachingP205C/205C.pdf http://psiexp.ss.uci.edu/research/teachingP205C/205C.pdf -- Somewhat unfinished but very good hands-on guide for a total novice David Draper "Bayesian Modeling, Inference and Prediction" https://users.soe.ucsc.edu/~draper/draper-BMIP-dec2005.pdf https://users.soe.ucsc.edu/~draper/draper-BMIP-dec2005.pdf -- Quite good if you're already familiar with a field, might be slightly daunting for a beginner and the classic, David MacKay's "Information Theory, Inference, and Learning Algorithms" http://www.inference.phy.cam.ac.uk/itprnn/book.html http://www.inference.phy.cam.ac.uk/itprnn/book.html
chmullig 11y ago
Stan is very much a successor to BUGS, although I'm not sure BUGS is better known outside certain social science research communities these days.
- lgas 11y ago
  It's not.
dewarrn1 11y ago
Another endorsement for Gelman's books — they're some of the best instructional statistics work that I've read.
- eruditely 11y ago
  You really think they're instructional? I would say it's sort of complete, but no means introductory.
dustintran 11y ago
Hi, stan dev here. I think viewing Stan as a better BUGS is helpful but limiting. The syntax is similar, but the class of models Stan fits is far more general. The class of algorithms we have available also goes beyond MCMC, e.g., variational inference, optimization, and interfaces to Stan exist on all primary programming languages. It's more helpful to think of Stan as its own probabilistic programming language, and arguably the biggest entity with the largest user base.
- thisisdave 11y ago
  >the class of models Stan fits is far more general. This is mostly true, but last time I checked, Stan still couldn't sample discrete variables like BUGS can. Stan can only fit models with discrete parameters (e.g. finite mixtures) if the programmer is smart enough to integrate them out.

tel 11y ago

+1 on all of this (except I'm a fan of full MCMC, have you used ADVI?)

probdist 11y ago
Haven't tried ADVI yet, but I think it is (likely) a great idea. In general automatic differentiation is a wildly powerful technique and is underutilized in computational science. Even in pure optimization problems people rarely leverage it to get J or H information as often as they probably should. I'll admit to not being super familiar with all the machinery under the hood in the ADVI paper [1]. It wasn't out when I started working on my current project. [1] http://arxiv.org/abs/1506.03431 http://arxiv.org/abs/1506.03431
- proditus 11y ago
  do let me know how it goes if you do try ADVI. we're also working on making it more robust to initialization and step-sizes. stay tuned.
  - mjw 11y ago
    AVDI is incredibly awesome. Thanks so much for this work. Really nice readable paper on it too. As someone working with large datasets I think automating variational inference + SGD to work with a broad class of models is really the way forwards and a big force multiplier for machine learning.

kristjankalm 11y ago

some useful books available as a PDF: David Barber "Bayesian Reasoning and Machine Learning" http://web4.cs.ucl.ac.uk/staff/D.Barber/pmwiki/pmwiki.php?n=Brml.HomePage http://web4.cs.ucl.ac.uk/staff/D.Barber/pmwiki/pmwiki.php?n=... -- Thorough and systematic, accompanied by a Matlab library Marc Steyvers "Computational Statistics with Matlab" http://psiexp.ss.uci.edu/research/teachingP205C/205C.pdf http://psiexp.ss.uci.edu/research/teachingP205C/205C.pdf -- Somewhat unfinished but very good hands-on guide for a total novice David Draper "Bayesian Modeling, Inference and Prediction" https://users.soe.ucsc.edu/~draper/draper-BMIP-dec2005.pdf https://users.soe.ucsc.edu/~draper/draper-BMIP-dec2005.pdf -- Quite good if you're already familiar with a field, might be slightly daunting for a beginner and the classic, David MacKay's "Information Theory, Inference, and Learning Algorithms" http://www.inference.phy.cam.ac.uk/itprnn/book.html http://www.inference.phy.cam.ac.uk/itprnn/book.html

chmullig 11y ago

Stan is very much a successor to BUGS, although I'm not sure BUGS is better known outside certain social science research communities these days.

lgas 11y ago
It's not.

dewarrn1 11y ago

Another endorsement for Gelman's books — they're some of the best instructional statistics work that I've read.

eruditely 11y ago
You really think they're instructional? I would say it's sort of complete, but no means introductory.

dustintran 11y ago

Hi, stan dev here. I think viewing Stan as a better BUGS is helpful but limiting. The syntax is similar, but the class of models Stan fits is far more general. The class of algorithms we have available also goes beyond MCMC, e.g., variational inference, optimization, and interfaces to Stan exist on all primary programming languages. It's more helpful to think of Stan as its own probabilistic programming language, and arguably the biggest entity with the largest user base.

thisisdave 11y ago
>the class of models Stan fits is far more general. This is mostly true, but last time I checked, Stan still couldn't sample discrete variables like BUGS can. Stan can only fit models with discrete parameters (e.g. finite mixtures) if the programmer is smart enough to integrate them out.