3 ms·

It's much more expensive to train models. Besides, compilers are not that smart yet. E.g. a HMM implemented in a PPL is far from the efficiency of hand-rolled c

by nextos 3y ago

It's much more expensive to train models. Besides, compilers are not that smart yet. E.g. a HMM implemented in a PPL is far from the efficiency of hand-rolled code. For many use cases, they are still a leaky abstraction. However, in areas where measuring uncertainty is important, they have taken off. Stan has become mainstream in Bayesian statistics. Pyro and PyMC are also quite used in industry (I have had recruiters contacting me for this skill). Infer.NET has its own niche on discrete and online inference. Infer.NET models ship with several Microsoft products. Other interesting PPLs include Turing.jl, Gen.jl, and the venerable BUGS.

rich_sasha 3y ago
I'd be curious as to what the specific end applications are, are there any you can share? I'm familiar with the tooling and played with it quite a bit... But never really figured out a practical application.
- nextos 3y ago
  Sure. It's hard to make this justice on a single comment, as there are lots of applications. Basically, any scenario with small or medium size datasets where you would use generative models. PPLs are just a way to encode generative models and get an inference engine compiled for you, instead of needing to write one. For example: * Landing the Apollo on the Moon or tracking systems used by e.g. Sidewinder employ Kalman filters. See Example 24.4 (p. 510) [1]. * Predicting ride demand on heavy-tailed time series. Uber does this all the time [2]. * Estimating the effect of some policy on data with hierarchical structures (State > County > Individual observations) [3]. [1] http://web4.cs.ucl.ac.uk/staff/D.Barber/textbook/200620.pdf http://web4.cs.ucl.ac.uk/staff/D.Barber/textbook/200620.pdf [2] https://pyro.ai/examples/forecasting_i.html https://pyro.ai/examples/forecasting_i.html [3] https://mc-stan.org/users/documentation/case-studies/radon_cmdstanpy_plotnine.html https://mc-stan.org/users/documentation/case-studies/radon_c...

by nextos 3y ago

rich_sasha 3y ago

I'd be curious as to what the specific end applications are, are there any you can share? I'm familiar with the tooling and played with it quite a bit... But never really figured out a practical application.

nextos 3y ago
Sure. It's hard to make this justice on a single comment, as there are lots of applications. Basically, any scenario with small or medium size datasets where you would use generative models. PPLs are just a way to encode generative models and get an inference engine compiled for you, instead of needing to write one. For example: * Landing the Apollo on the Moon or tracking systems used by e.g. Sidewinder employ Kalman filters. See Example 24.4 (p. 510) [1]. * Predicting ride demand on heavy-tailed time series. Uber does this all the time [2]. * Estimating the effect of some policy on data with hierarchical structures (State > County > Individual observations) [3]. [1] http://web4.cs.ucl.ac.uk/staff/D.Barber/textbook/200620.pdf http://web4.cs.ucl.ac.uk/staff/D.Barber/textbook/200620.pdf [2] https://pyro.ai/examples/forecasting_i.html https://pyro.ai/examples/forecasting_i.html [3] https://mc-stan.org/users/documentation/case-studies/radon_cmdstanpy_plotnine.html https://mc-stan.org/users/documentation/case-studies/radon_c...