2 ms·

I also did a PhD on MCMC (for sampling images), and agree fully with everything you said. I'll just add that I never felt I could trust HMC at all (I hardly use

I also did a PhD on MCMC (for sampling images), and agree fully with everything you said. I'll just add that I never felt I could trust HMC at all (I hardly used it) because it was too abstract, I couldn't form much intuition about why it mightn't be mixing. On the other hand I could easily understand the deficiencies of Gibbs sampling (in my case, changing one pixel at a time) and work around them (somewhat). I came away thinking that a 'feed-forward' approximation with a fixed amount of computation (e.g. a probabilistic neural net) would have been a much superior method to MCMC.

boulos 4y ago
> superior method to MCMC Superior in what ways? I could imagine simplicity for many problems. Roughly "here, replace your complex simulation with this magic function approximator". But for problems where you're calculating a physical quantity with a true answer, I don't see NNs being "much superior".
- versteegen 4y ago
  In my case I could decide how to model the thing I was trying to emulate (not simulate). Using models that are inherently difficult to sample from was not a great idea.