6 ms·
Probabilistic Programming
- adamnemecek 10y ago"Design and Implementation of Probabilistic Programming Languages" http://dippl.org http://dippl.org is pretty dank too. You implement your own probabilistic language in JS.
- eli_gottlieb 10y agoMy kingdom for a compiled, maintained PPL with recursive queries!
- piemonkey 10y agoAs a researcher that is currently working on implementing and improving existing PPLs, I'm very curious: what is your specific use-case that you have in mind for recursive queries?
- YeGoblynQueenne 10y ago>> what is your specific use-case that you have in mind for recursive queries? They probably want to run a recursive query. (sorry)
- posterboy 10y agoI never liked this approach to recursion, as in google's joke search correction for recursion, because recursion is a form of progressive iteration, not reiteration.
- YeGoblynQueenne 10y agoRecursion is frequently used as progressive iteration, but it can go further than that. Typical example, a factorial function, in Erlang: factorial(N) -> factorial(N,1). factorial(0, F) -> F; factorial(N, F) -> factorial(N-1, N*F). It's strange to read that as iteration. Instead it feels natural to describe it as composition from parts that are of the same kind as the whole.
- posterboy 10y agoof course that is some kind of iteration because tail recursion is isomorphic to loops and that N-1 sure looks like a while(--N)
- YeGoblynQueenne 10y agoI'm not sure I know what "isomorphic" means in this context. For sure, (tail) recursion places a pointer to a function on the stack while a loop is a conditional jump. They're not the same thing. I think the similarity between n-1 and --n is a red herring, here.
- eli_gottlieb 10y agoHonestly, I'd like to be able to try out some of the old social-reasoning demos from forestdb. It's not really a high-priority thing; it just bugs me that there were working languages for doing this (Church) and now there's basically just Anglican.
- wkv 10y agoIf you're interested in social reasoning and nested inference models, there's some examples of how they're implemented in WebPPL here: http://agentmodels.org/chapters/7-multi-agent.html http://agentmodels.org/chapters/7-multi-agent.html. I think many of the reasoning examples from forestdb that are currently written in Church could be translated into WebPPL in a fairly straightforward manner using the same structure as the agentmodels examples.
- gnipgnip 10y agoCan someone in the know please comment about how this is useful ? I mean yes this is neat, but inference engines already work declaratively with factor-graphs and potentials. Are there techniques exploiting algebra/logic that helps make inference faster in practice ?
- kqr 10y agoIf this site is talking about what I think it is, then "regular" programming languages are a special case of this where the probability distribution of each value is the trivial case. Only based on that, this will be useful for anything regular programming is useful. In addition, this makes modeling uncertainty much easier (in the sense of "closer at hand"). That may allow for new ways of dealing with user input. Instead of saying "is this email address valid or invalid", we can start asking questions like "Is the probability of this email being valid larger than 99%? Then we'll accept it. Is it larger than 95%? Then we'll ask the user to confirm it. Is it less than 95%? Then we'll tell the user it's incorrect and have them retype it." These are things we normally don't care to model because it would require lots of additional machinery. With that machinery built into the programming language, it is much easier to reach for, potentially with a better user experience to boot.
- gnipgnip 10y agoYes, this is precisely what is called probabilistic inference. I was trying to find out what more it can do other than parsing a DSL into a graphical model.
- kqr 10y agoPlenty of things! The real world is often best modeled by a probabilistic model; you can rarely state facts in definite terms. The probability the bus arrives on time may be 70%. The probability that a particular host is up may be 92%. The probability of needing a reboot within the next week is 2%. The probability that the user switched off Wifi on their machine is 4%, and the probability that their router is having trouble is 3%. These numbers can be based on actual measurements. What do we tell the user when they have problems connecting? Instead of just saying "it's either this or that" we can run the numbers, and perhaps in aggregate there's an overwhelming probability it's a particular event, in which case we suggest that first. There are so many cases where we don't actually know for certain all the parameters involved, but the conventional approach is still to round the probability either up to 100% or down to 0%. Simply because that's easier in conventional programming languages. As a result, you might not see these events as having probability distributions, but they do.
- thecity2 10y agoI think "Bayesian Inferential Programming" would be more direct terminology.
- ptero 10y agoI know nothing about the probabilistic programming, but would not it be easier (and more applicable) to have a library with relevant functionality and a bunch of interfaces for Go/C/Python/... than to implement a new language? An honest question.
- jbpetersen 10y agoExample from another comment here: http://dippl.org/chapters/02-webppl.html http://dippl.org/chapters/02-webppl.html
- glassx 10y agoIt seems that their definition of "language" encompasses more than general-purpose PLs and includes DSLs, libraries and code-generators. In their list there are libraries for Scala, .NET, Python, C, OCaml and Javascript.
- ppcsf 10y agoI developed a PPL within C#, which I guess has a more familiar syntax for a lot of people. Basically, PPLs allow you to treat entire probabilistic models as primitives, and combine them as easily as we add integers. Let's say we have two models describing the distribution of university grades in America and India, and we want to combine them into a single model, where you have a 25% chance of being an American, and a 75% chance of being an Indian: var combined = from isAmerican in Bernoulli(0.25) from grade in isAmerican ? americanGrades : indianGrades select grade; This is now a new model that can be composed further with other models. Building models like this feels very expressive. The inference method is also completely decoupled from the model specification process, allowing us to perform a Sequential Monte Carlo just by writing: var smcResults = combined.SmcMultiple(100); or var pimhResults = combined.Pimh(10); for a Particle-Independant Metropolis-Hastings. There's more examples here: https://github.com/joashc/csharp-probability-monad https://github.com/joashc/csharp-probability-monad
- tbenst 10y agoI attended DARPA's Probabilistic Programming for Advancing Machine Learning (PPAML) summer school. The aim of Probabilistic Programming languages (PPL) is to abstract away the act of Bayesian inference into modular engines such that switching from say Hamiltonian Monte Carlo to a Particle Filter requires changing exactly one string. If you can write a model in a PPL, you get inference for free [1]. The purported aim is to allow machine learning code that today requires 1000-10,000 lines of code to be written in 10-100 lines. This works brilliantly for single shot learning. Let's say you are trying to teach a computer to recognize a handwritten character after a single example. First, you build a generative model that follows how letters are constructed: hand touches paper and makes a primitive shape (line, curve, loop, etc.). Multiple such primitives are strung together to form a character. For each example characters, infer the most likely sequence of such primitives. When a new classification is requested, take samples from the generative model for each known character. Calculate the difference in pixel value, and do this hundreds of thousands of times. You can now construct an accurate marginal probability for each known character while only needing a single example. Powerful stuff! [1] efficiency not guaranteed..(yet) Edit: Here's a few curated resources: http://webppl.org/ http://webppl.org/ http://www.robots.ox.ac.uk/~fwood/anglican/ http://www.robots.ox.ac.uk/~fwood/anglican/ http://mc-stan.org/ http://mc-stan.org/ http://dippl.org/ http://dippl.org/ https://probmods.org/ https://probmods.org/
- gnipgnip 10y agoThe "Inverse Graphics" paper is a nice read too :) http://www.cv-foundation.org/openaccess/content_cvpr_2015/html/Kulkarni_Picture_A_Probabilistic_2015_CVPR_paper.html http://www.cv-foundation.org/openaccess/content_cvpr_2015/ht...
- adamnemecek 10y agoWhat are the current ppl languages that are pushing the proverbial envelope?
- tbenst 10y agoAnglican for high performance (Clojure host, strong multithreading support) and WebPPL for ease of integration (JavaScript client side or server side). Both are implemented using continuation passing style trampolines.
- dorianm 10y agoThere is a "probabilistic programming" gem for ruby: https://github.com/urso/rb_probdsl https://github.com/urso/rb_probdsl (from 2010)
- murbard2 10y agoAs much as I love the concept probabilistic programming, the name makes me cringe every time I hear it. It’s verbose, doesn’t abbreviate well and borders on being a tongue twister. More substantially, the name suggests that the act of programming itself is probabilistic (a real but distinct phenomenom :) ) Worse, rather than highlighting the key value proposition: automated inference, the name suggests a focus on the development of probabilistic programs. Perhaps, as a result, many introductions to probabilistic programming concern themselves almost exclusively on the comparatively trivial task of expressing models as algorithms while blissfully ignoring the thorny problem of inference. I suggest talking about “automated inference” instead. It rolls off the tongue much more easily, and, more importantly, gets to the heart of the matter.
- bogomipz 10y agoIf anyone is interested Manning released a decent book on the subject a few months back. You can download a couple of sample chapters here: https://www.manning.com/books/practical-probabilistic-programming https://www.manning.com/books/practical-probabilistic-progra...