3 ms·
I know they've traditionally been quite fiddly, but I'm pretty sure computers can be persuaded to help derive the maths for variational methods these days. Per
by mjw 11y ago
I know they've traditionally been quite fiddly, but I'm pretty sure computers can be persuaded to help derive the maths for variational methods these days.
Perhaps a more important difference is that MCMC, while slow, is exact in the limit. Variational methods won't converge to the true posterior no matter how long you run them. You'll converge to an approximate answer which depends on the particular variational form you choose to use.
- x0x0 11y agoAre you aware of any work (or researchers) working on that? I would be very interested. And I think your second sentence reinforces my point -- it makes variational methods either more fiddly, or require more understanding to use well. edit: here's one such (limited but nice) effort: http://ebonilla.github.io/papers/nguyen-bonilla-nips-2014.pdf http://ebonilla.github.io/papers/nguyen-bonilla-nips-2014.pd...
- davmre 11y agoMicrosoft's Infer.NET is essentially an automated system for variational (/expectation-propagation) inference. It implements primitives for common operations and distributions, and then uses the local structure of mean-field inference ("variational message passing": http://www.jmlr.org/papers/volume6/winn05a/winn05a.pdf http://www.jmlr.org/papers/volume6/winn05a/winn05a.pdf) to build up variational inference algorithms for arbitrary factor graphs. It's not infinitely flexible and doesn't solve all problems related to variational inference, but once you get used to it you can iterate quite quickly on model refinements, inference tweaks, etc. without any tedious derivations.
- x0x0 11y agothank you!