3 ms·
> probabilistic graphical models- of which transformers is an example Having done my PhD in probabilistic programming... what?
by eli_gottlieb 1y ago
> probabilistic graphical models- of which transformers is an example
Having done my PhD in probabilistic programming... what?
- pishpash 1y agoIt's got nothing to do with PGM's. However, there is the flavor of describing graph structure by soft edge weights vs. hard/pruned edge connections. It's not that surprising that one does better than the other, and it's a very obvious and classical idea. For a time there were people working on NN structure learning and this is a natural step. I don't think there is any breakthrough here, other than that computation power caught up to make it feasible.
- dekhn 1y agoI was talking about things inspired by (for example) hidden markov models. See https://en.wikipedia.org/wiki/Graphical_model https://en.wikipedia.org/wiki/Graphical_model In biology, PGMs were one of the first successful forms of "machine learning"- given a large set of examples, train a graphical model using probabilities using EM, and then pass many more examples through the model for classification. The HMM for proteins is pretty straightforward, basically just a probabilistic extension of using dynamic programming to do string alignment. My perspective- which is a massive simplification- is that sequence models are a form of graphical model, although the graphs tend to be fairly "linear" and the predictions generate sequences (lists) rather than trees or graphs.