Lack of good priors is definitely a weak spot for BNNs. I also like the concept of Generalization-Agnostic Priors. But... > So viewed through this lens, BNNs w

Lack of good priors is definitely a weak spot for BNNs. I also like the concept of Generalization-Agnostic Priors. But... > So viewed through this lens, BNNs with arbitrary priors are nothing more than an architectural decision. A BNN is just a neural network that maps its input to a distribution over outputs; the prior is just a hyperparameter of the model. Just making the network Bayesian bought us nothing. It will only be helpful if we find a good prior, and validate that we are actually doing accurate inference. If you personally believe that exploring this space of priors (similar to exploring the space of architectures or hyperparameters) is particularly promising, then that is a good reason to keep working on BNNs. Isn't this overstating it a bit... Making the network Bayesian (even if the priors are generalization-agnostic) still bought us something: after training we have a posterior distribution over weights that contains both weights that generalize and weights that don't generalize. That's better than being unlucky and only having weights that don't generalize at the end of training. I'm not saying this advantage is easy to achieve in practice, or benefit from. But it's not nothing. Also Bayesian inference as a framework has a distinct advantage over the usual aimless architectural twiddling that so much "research" seems to focus on: it's intellectually/mathematically sound.