In the context of Bayesian inference, the KL divergence between the prior p(m) and the posterior p(m|x) is the information gained by observing x. With VAEs, ad

In the context of Bayesian inference, the KL divergence between the prior p(m) and the posterior p(m|x) is the information gained by observing x. With VAEs, adding a KL divergence to the loss term can be thought of as regularizing information gain from individual inputs.