5 ms·
Metropolis Algorithm and Bayesian posterior sampling
- panic 10y agoMetropolis Light Transport (http://graphics.stanford.edu/papers/metro/ http://graphics.stanford.edu/papers/metro/) is a clever rendering technique based on this sampling technique applied to light paths.
- boulos 10y agoOne thing people (myself included!) often missed though is that Metropolis is extremely prone to flicker. I think the best summary of why comes from Kalos and Whitlock (2nd Edition): > The M(RT)2 algorithm is very simple and powerful; it can be used to sample essentially any distribution function regardless of analytic complexity in any number of dimensions. Complementary disadvantages are that sampling is correct only asymptotically and that successive variables produced are correlated, often very strongly. This means that the evaluation of integrals normally produces positive correlations in the values of the integrand, with consequent increase in variance for a fixed number of steps as compared with independent samples. Also the method is not well suited to sampling distributions with parameters that change frequently.
- erichocean 10y ago> One thing people (myself included!) often missed though is that Metropolis is extremely prone to flicker. Doing PCA across frames basically eliminates that. IIRC Pixar did a paper on the technique.
- jcl 10y agoActually, the Renderman denoiser is based on work from Disney Research. https://renderman.pixar.com/view/denoiser https://renderman.pixar.com/view/denoiser PCA would be one way to do that. This recent paper suggests that they may be decomposing the samples mostly by path type, though: https://www.disneyresearch.com/publication/pathspace-decomposition/ https://www.disneyresearch.com/publication/pathspace-decompo...
- boulos 10y agoHe's referring to the older 2008 paper that did PCA, Statistical Acceleration for Animated Global Illumination: http://graphics.pixar.com/library/ShotRendering/paper.pdf http://graphics.pixar.com/library/ShotRendering/paper.pdf
- jcl 10y agoI found this tutorial was a good overview for both the Metropolis algorithm and for Metropolis Light Transport: https://facweb.cs.byu.edu/graphics/publications/metropolisTutorial.pdf https://facweb.cs.byu.edu/graphics/publications/metropolisTu...
- gaur 10y agoWhat's the rationale behind 100 as the thinning factor? Just a conservative number based on the autocorrelation length?
- imurray 10y agoThe thinning here is based on a misconception. If the purpose is plotting a histogram it doesn't matter if the samples are correlated. The bin heights are consistently estimated if you keep everything. Throwing away intermediate steps usually just introduces noise. Thinning is a good idea if the samples are strongly correlated, and the computation to process them all would be better spent on running the chain for longer. Or if you don't know what computation you want in advance, and you can't afford to store everything. (Also, as someone points out in the comments, the implementation of the Metropolis algorithm in this post is wrong.)
- asifjamil 10y agoCan anyone here recommend a good reference (blog, e-book, etc.) for beginners in Bayesian statistical analysis?
- vitohuang 10y agoYou can have look at this https://github.com/CamDavidsonPilon/Probabilistic-Programming-and-Bayesian-Methods-for-Hackers https://github.com/CamDavidsonPilon/Probabilistic-Programmin...
- dragandj 10y agoThe only book accessible by beginners, while still dealing with real-world level analysis: https://www.amazon.com/Doing-Bayesian-Data-Analysis-Second/dp/0124058884/ https://www.amazon.com/Doing-Bayesian-Data-Analysis-Second/d...
- stared 10y agoBook: - Information Theory, Inference and Learning Algorithms Textbook by David J. C. MacKay (free pdf: http://www.inference.phy.cam.ac.uk/itila/ http://www.inference.phy.cam.ac.uk/itila/) Other things: - (the already mentioned http://camdavidsonpilon.github.io/Probabilistic-Programming-and-Bayesian-Methods-for-Hackers/ http://camdavidsonpilon.github.io/Probabilistic-Programming-...) - https://github.com/markdregan/Bayesian-Modelling-in-Python https://github.com/markdregan/Bayesian-Modelling-in-Python (a practical introduction to PyMC3)
- kgwgk 10y agoStatistical Rethinking (the lectures are available on youtube): http://xcelab.net/rm/statistical-rethinking/ http://xcelab.net/rm/statistical-rethinking/
- triplepoint217 10y agoBayesian Logical Data Analysis for physical scientists by Phil Gregory is pretty good, particularly if you are some kind of scientist. source: I TA'd a course in Data analysis, primarily bayesian, that used that as the textbook.