3 ms·
MCMC probabilistic programming won't be the next big thing in machine learning though. You need some sort of maximum likelihood / maximum a posteriori / variati
by jules 12y ago
MCMC probabilistic programming won't be the next big thing in machine learning though. You need some sort of maximum likelihood / maximum a posteriori / variational bayes to do machine learning. MCMC is far too slow.
- tel 12y agoMany of these PPLs use variational Bayes.
- eli_gottlieb 12y agoAt least according to my reading, fast algorithms for calculating expectations and modes via sampling and approximation actually came before the ones for doing generalized probabilistic inference via sampling and approximation.
- murbard2 12y agoMaybe not. Not all probabilistic programming relies on MCMC, you can do MAP with probabilistic programming too. But in general, it's true that it tends towards Bayesian integration. In a sense, MCMC tries to solve #P complete problems, while MAP "only" tries to solve NP complete problems. The former are probably intrinsically harder than the latter. However, it's quite possible that some clever implementation might yield results which are as good as MAP in the same amount of time, and progressively refine them with more time. There are two free lunches with the curse of dimensionality: - the manifold hypothesis - Monte-Carlo integration True, the dimension rears its head as an ugly multiplicative constant in the latter - still, there has to be something there.