4 ms·

This is the next breakthrough coming up after Lisp...

by avaku 12y ago

eli_gottlieb 12y ago
Probabilistic programming is already basically the definitive Next Big Thing in machine learning, since it allows us to automate reasoning over far more sophisticated models than we could previously write down by hand, while the approximate inference techniques being used make numerical evaluation of Bayesian reasoning tractable. The generality and power of the approach are, frankly, amazing.
- avaku 12y ago
  Yes, but it's not widely commercial yet. Huge startup opportunities! We are thinking about it. If wanna get in touch find my contact via http://akuz.me http://akuz.me (my personal website)
- jules 12y ago
  MCMC probabilistic programming won't be the next big thing in machine learning though. You need some sort of maximum likelihood / maximum a posteriori / variational bayes to do machine learning. MCMC is far too slow.
  - tel 12y ago
    Many of these PPLs use variational Bayes.
  - eli_gottlieb 12y ago
    At least according to my reading, fast algorithms for calculating expectations and modes via sampling and approximation actually came before the ones for doing generalized probabilistic inference via sampling and approximation.
  - murbard2 12y ago
    Maybe not. Not all probabilistic programming relies on MCMC, you can do MAP with probabilistic programming too. But in general, it's true that it tends towards Bayesian integration. In a sense, MCMC tries to solve #P complete problems, while MAP "only" tries to solve NP complete problems. The former are probably intrinsically harder than the latter. However, it's quite possible that some clever implementation might yield results which are as good as MAP in the same amount of time, and progressively refine them with more time. There are two free lunches with the curse of dimensionality: - the manifold hypothesis - Monte-Carlo integration True, the dimension rears its head as an ugly multiplicative constant in the latter - still, there has to be something there.

This is the next breakthrough coming up after Lisp...

by avaku 12y ago

This is the next breakthrough coming up after Lisp...

Probabilistic programming is already basically the definitive Next Big Thing in machine learning, since it allows us to automate reasoning over far more sophisticated models than we could previously write down by hand, while the approximate inference techniques being used make numerical evaluation of Bayesian reasoning tractable. The generality and power of the approach are, frankly, amazing.

avaku 12y ago
Yes, but it's not widely commercial yet. Huge startup opportunities! We are thinking about it. If wanna get in touch find my contact via http://akuz.me http://akuz.me (my personal website)
jules 12y ago
MCMC probabilistic programming won't be the next big thing in machine learning though. You need some sort of maximum likelihood / maximum a posteriori / variational bayes to do machine learning. MCMC is far too slow.
- tel 12y ago
  Many of these PPLs use variational Bayes.
- eli_gottlieb 12y ago
  At least according to my reading, fast algorithms for calculating expectations and modes via sampling and approximation actually came before the ones for doing generalized probabilistic inference via sampling and approximation.
- murbard2 12y ago
  Maybe not. Not all probabilistic programming relies on MCMC, you can do MAP with probabilistic programming too. But in general, it's true that it tends towards Bayesian integration. In a sense, MCMC tries to solve #P complete problems, while MAP "only" tries to solve NP complete problems. The former are probably intrinsically harder than the latter. However, it's quite possible that some clever implementation might yield results which are as good as MAP in the same amount of time, and progressively refine them with more time. There are two free lunches with the curse of dimensionality: - the manifold hypothesis - Monte-Carlo integration True, the dimension rears its head as an ugly multiplicative constant in the latter - still, there has to be something there.