3 ms·
There are Bayesian neural networks that could apparently track probability rather than just e.g. randomly selecting one output from the top-k based on probabili
by hnuser123456 10mo ago
There are Bayesian neural networks that could apparently track probability rather than just e.g. randomly selecting one output from the top-k based on probability, but I'm still learning up on them myself. Sounds like they're not normally combined with language models.
- inkysigma 10mo agoIirc, the problem with Bayesian neural networks is that they're significantly more difficult to train. Using stuff like SVI reduces a lot of the representational ability of the distribution over weights. It's also questionable how useful the uncertainty over weights is. I suppose in the tradition of Bayesian influence, VAEs and the like are still common though.