4 ms·

Sure. I will also continue to answer any questions others have about these systems on this thread.

by cf 10y ago

nextos 10y ago
What work has been done in optimizing code / sampling? My understanding is that if I write e.g. an HMM in any of these languages, the resulting code will be suboptimal in performance and accuracy vs e.g. Viterbi.
- cf 10y ago
  So there has been work done in compiling probabilistic programs directly into C or C++ [1]. Your intuition is correct that in many of these languages if you wrote an HMM and did MAP inference over it you wouldn't recover the Viterbi algorithm, but there is some code in WebPPL[2] that handles this case as outlined in https://arxiv.org/abs/1206.3555 https://arxiv.org/abs/1206.3555. [1] https://web.stanford.edu/~ngoodman/papers/aistats2014-shred.pdf https://web.stanford.edu/~ngoodman/papers/aistats2014-shred.... [2] http://docs.webppl.org/en/master/inference/methods.html#enumeration http://docs.webppl.org/en/master/inference/methods.html#enum...
foob 10y ago
It seems like sampling based inference approaches are the most common. Could you possibly comment on the current state of the art regarding more analytical approaches? I sometimes deal with problems where it's possible to find closed form solutions but they're very painstaking to find by hand and can be extremely complicated. It would be incredible if a PPL compiler were able to do this grunt work and then produce code that could compute desired quantities directly.
- cf 10y ago
  There are two lines of research in this direction. So there is work from my lab [1] and some folks at ETH-Zurich [2] is automatically finding closed form solutions. These when used do give performance equivalent to handwritten methods. Beyond that we are seeing some cool stuff with Blackbox Variational Inference [3] and other Automatic Variational [4] solvers coming out of Blei's lab. At present these capabilities are spread about the different system but I expect all them to eventually end up available in whichever you choose to use. [1] http://homes.soic.indiana.edu/ccshan/rational/simplify-padl.pdf http://homes.soic.indiana.edu/ccshan/rational/simplify-padl.... [2] http://www.srl.inf.ethz.ch/papers/psi-solver.pdf http://www.srl.inf.ethz.ch/papers/psi-solver.pdf [3] http://www.cs.columbia.edu/~blei/papers/RanganathGerrishBlei2014.pdf http://www.cs.columbia.edu/~blei/papers/RanganathGerrishBlei... [4] http://arxiv.org/pdf/1603.00788v1.pdf http://arxiv.org/pdf/1603.00788v1.pdf

Sure. I will also continue to answer any questions others have about these systems on this thread.

by cf 10y ago

Sure. I will also continue to answer any questions others have about these systems on this thread.

What work has been done in optimizing code / sampling? My understanding is that if I write e.g. an HMM in any of these languages, the resulting code will be suboptimal in performance and accuracy vs e.g. Viterbi.

cf 10y ago
So there has been work done in compiling probabilistic programs directly into C or C++ [1]. Your intuition is correct that in many of these languages if you wrote an HMM and did MAP inference over it you wouldn't recover the Viterbi algorithm, but there is some code in WebPPL[2] that handles this case as outlined in https://arxiv.org/abs/1206.3555 https://arxiv.org/abs/1206.3555. [1] https://web.stanford.edu/~ngoodman/papers/aistats2014-shred.pdf https://web.stanford.edu/~ngoodman/papers/aistats2014-shred.... [2] http://docs.webppl.org/en/master/inference/methods.html#enumeration http://docs.webppl.org/en/master/inference/methods.html#enum...

foob 10y ago

It seems like sampling based inference approaches are the most common. Could you possibly comment on the current state of the art regarding more analytical approaches? I sometimes deal with problems where it's possible to find closed form solutions but they're very painstaking to find by hand and can be extremely complicated. It would be incredible if a PPL compiler were able to do this grunt work and then produce code that could compute desired quantities directly.

cf 10y ago
There are two lines of research in this direction. So there is work from my lab [1] and some folks at ETH-Zurich [2] is automatically finding closed form solutions. These when used do give performance equivalent to handwritten methods. Beyond that we are seeing some cool stuff with Blackbox Variational Inference [3] and other Automatic Variational [4] solvers coming out of Blei's lab. At present these capabilities are spread about the different system but I expect all them to eventually end up available in whichever you choose to use. [1] http://homes.soic.indiana.edu/ccshan/rational/simplify-padl.pdf http://homes.soic.indiana.edu/ccshan/rational/simplify-padl.... [2] http://www.srl.inf.ethz.ch/papers/psi-solver.pdf http://www.srl.inf.ethz.ch/papers/psi-solver.pdf [3] http://www.cs.columbia.edu/~blei/papers/RanganathGerrishBlei2014.pdf http://www.cs.columbia.edu/~blei/papers/RanganathGerrishBlei... [4] http://arxiv.org/pdf/1603.00788v1.pdf http://arxiv.org/pdf/1603.00788v1.pdf