3 ms·

Has probabilistic programming been shown to solve problems better than machine learning or deep learning approaches? I remember it being pretty hyped 5-6 years

Has probabilistic programming been shown to solve problems better than machine learning or deep learning approaches? I remember it being pretty hyped 5-6 years ago …

vibrio_bobtail 5y ago
I wouldn't contrast it with machine learning or deep learning. Probabilistic programming is focused on building languages or libraries that incorporate fundamental probabilistic building blocks, model building statements and inference strategies as first class citizens within the language. Its been remarkably successful, check out Stan, Pyro, PyMC, Tensorflow Probability, JAGS, BUGS and Turing as very successful projects that have been used to tackle challenging and diverse problems with probabilistic modeling . The more modern probabilistic programming languages are actually designed to incorporate the advances of deep learning by making it easy to embed ANNs into models by using them to parameterize random variables, Pyro and Tensorflow Probability are probably the best examples of this.
medo-bear 5y ago
note that these are not exclusive. you could divide ML into a traditional statistical approach and a probabilistic one that is concerned with deriving the underlying probability distribution. probabilistic programming is kind of like a domain specific language for achieving this. there is also differential programming that works on the same principle. there are certainly industrial usages of this paradigm. look up pyro (http://pyro.ai/examples/intro_part_i.html http://pyro.ai/examples/intro_part_i.html) for ppl and jax (https://github.com/google/jax https://github.com/google/jax) for differential programming
nerdponx 5y ago
It's the standard tool for building Bayesian statistical models, which aren't glamorously crushing SotA records on AI-like prediction tasks, but are valuable in both academia and industry.
horse911 5y ago
Not an either-or situation. Many ML algorithms can be formulated in a probabilistic way, and PPLs can represent them. But the choice of the underlying inferential algorithm (MCMC, variational, etc) greatly affects the scale and type of solvable problems, so it's not just the language itself. PPLs using MCMC (e.g., Stan and pymc) are first-choice tools for Bayesian inference on small to medium data analysis, which is lots more common than Google-scale data analysis.
vlovich123 5y ago
My professor in school years ago was working on probabilistic hardware and showing pretty good thermal benefits. It’s a drastically different way to build software so I don’t know how commercially viable it’ll become until someone shows it has a drastically better power per watt ratio (like 1000x or 10000x, maybe higher to account for the cost of transitioning existing software)
- TaylorPhebillo 5y ago
  I'd be interested in learning more about that if you could share a name of the professor or project?
  - vlovich123 5y ago
    Dr Todd L. Veldhuizen. I looked through his research and don’t see anything there so either I’m misremembering or the research hadn’t gone anywhere (or still incomplete).

Has probabilistic programming been shown to solve problems better than machine learning or deep learning approaches? I remember it being pretty hyped 5-6 years ago …

vibrio_bobtail 5y ago

I wouldn't contrast it with machine learning or deep learning. Probabilistic programming is focused on building languages or libraries that incorporate fundamental probabilistic building blocks, model building statements and inference strategies as first class citizens within the language. Its been remarkably successful, check out Stan, Pyro, PyMC, Tensorflow Probability, JAGS, BUGS and Turing as very successful projects that have been used to tackle challenging and diverse problems with probabilistic modeling . The more modern probabilistic programming languages are actually designed to incorporate the advances of deep learning by making it easy to embed ANNs into models by using them to parameterize random variables, Pyro and Tensorflow Probability are probably the best examples of this.

medo-bear 5y ago

note that these are not exclusive. you could divide ML into a traditional statistical approach and a probabilistic one that is concerned with deriving the underlying probability distribution. probabilistic programming is kind of like a domain specific language for achieving this. there is also differential programming that works on the same principle. there are certainly industrial usages of this paradigm. look up pyro (http://pyro.ai/examples/intro_part_i.html http://pyro.ai/examples/intro_part_i.html) for ppl and jax (https://github.com/google/jax https://github.com/google/jax) for differential programming

nerdponx 5y ago

It's the standard tool for building Bayesian statistical models, which aren't glamorously crushing SotA records on AI-like prediction tasks, but are valuable in both academia and industry.

horse911 5y ago

Not an either-or situation. Many ML algorithms can be formulated in a probabilistic way, and PPLs can represent them. But the choice of the underlying inferential algorithm (MCMC, variational, etc) greatly affects the scale and type of solvable problems, so it's not just the language itself. PPLs using MCMC (e.g., Stan and pymc) are first-choice tools for Bayesian inference on small to medium data analysis, which is lots more common than Google-scale data analysis.

vlovich123 5y ago

My professor in school years ago was working on probabilistic hardware and showing pretty good thermal benefits. It’s a drastically different way to build software so I don’t know how commercially viable it’ll become until someone shows it has a drastically better power per watt ratio (like 1000x or 10000x, maybe higher to account for the cost of transitioning existing software)

TaylorPhebillo 5y ago
I'd be interested in learning more about that if you could share a name of the professor or project?
- vlovich123 5y ago
  Dr Todd L. Veldhuizen. I looked through his research and don’t see anything there so either I’m misremembering or the research hadn’t gone anywhere (or still incomplete).