3 ms·
A heuristic guide...If you have more data than knowledge about the domain, and you want to do classification/prediction, then NNs are a good. PGMs are good if
by mamp 10y ago
A heuristic guide...If you have more data than knowledge about the domain, and you want to do classification/prediction, then NNs are a good.
PGMs are good if you have knowledge that is important to encode; you need modularity i.e. you don't want to embed priors in the model; you have strong causal relationships e.g. diagnostic models with 'explaining away'; you want to integrate value of information e.g. what test should I do next to resolve uncertainty.
You can mix ML with knowledge in PGMs. The downside is the computational complexity of inference which is NP-hard for both exact and approximate inference, although you can identify the complexity of the model at design time and make the appropriate modelling trade-offs.
- wvbeaaoo 10y agoby complexity of inference do you mean the complexity of learning the structure of a model ? because inference on an existing PGM is linear in the number of edges with belief-propagation isn't it ?
- mamp 10y agoIt's linear in singly connected networks, not for multiply connected graphs. However NP-hard is the worst case as I mentioned (p288 Koller & Friedman)
- wvbeaaoo 10y agoby multiply connected graphs do you mean graphs with cycles ?
- mamp 10y agoYes, although in directed acyclic graphs the 'cycles' manifest as multiple paths to a node.
- wvbeaaoo 10y agocycles can be handled in two ways: if you are happy with approximate solutions, loopy BP can give that (still linear, but may take longer and there's parameter tuning), for exact solutions you can rewrite the graph to "carry" dependencies (latest paper by Frey)
- Xcelerate 10y agoCould you link to that paper? And does it have anything to do with the junction tree algorithm?
- wvbeaaoo 10y agohttp://www.psi.toronto.edu/~psi/pubs2/1999%20and%20before/134.pdf http://www.psi.toronto.edu/~psi/pubs2/1999%20and%20before/13... I don't know about junction trees but it probably connects as junction trees are a generalization of factor graphs
- Xcelerate 10y agoThanks!
- wvbeaaoo 10y agothe Christopher Bishop chapter on graphical models has a good section on junction trees IIRC