3 ms·

It's cool how "bayesian neural networks define a distribution over neural networks" –– so you can "sample from the posterior." Reminds me of the Gaussian Proces

It's cool how "bayesian neural networks define a distribution over neural networks" –– so you can "sample from the posterior." Reminds me of the Gaussian Process learning framework, which seems quite similar (distributions over functions). Has anyone thought of this overlap? edit: posted before reading other comments; seems like @syntaxing's link is what I'm looking for: "A network with infinitely many weights with a distribution on each weight is a Gaussian process"

clircle 8y ago
That a Bayesian NN "defines a distribution over neural networks" means that a Bayesian NN is a NN with priors on the weights. So it's not that there is some special similarity to GPs. BNNs are like any other Bayesian model, but now a NN is the likelihood. The situation is usually flipped with Bayesian GP models -- a GP is usually used as a prior on the linear predictor.