The idea of some of these packages (Blog, BUGS, Church, HBC) is that you describe how you would generate synthetic random data if you had to. If you don’t know

The idea of some of these packages (Blog, BUGS, Church, HBC) is that you describe how you would generate synthetic random data if you had to. If you don’t know any intermediate quantities, you specify distributions from which plausible values could be drawn. This is a way of defining your model and your uncertainties in a way that is easy to critique (“Your synthetic data looks rubbish!”). Then given some observed images, complete with noise, the systems can infer what the underlying quantities that generated the image were. A simple example: “is that smudge of pixels one star or two, and how probable is each option?” Some of the packages (Alchemy, Factorie) ask you to define joint probability distributions differently. Rather than asking for a sequence of steps that would sample from the distribution, you specify factors or soft constraints amongst variables. The product of the factors evaluated at a setting of the variables gives its probability up to a (often hard-to-compute) constant. The point of these packages is also to infer interesting quantities given observed data. The factors may have unknown parameters, which can be fitted to the data (“learning”). In the first set of packages unknown parameters are just variables, infered along with everything else. The first group of packages let you define directed graphical models, the sampling schemes can be defined on a directed acyclic graph. Factor graphs, used by the second group, can be defined on undirected graphs. Infer.NET lets you express directed models, but internally for algorithmic reasons it converts to undirected factor graphs. PMTK while not finished, seems to be aiming to provide special support for a variety of machine learning techniques, in addition to generic graphical model support. Dyna doesn't really fit in with the rest. It is a programming language that makes dynamic programming easy, which is useful in some inference tasks.