In NN training, minimizing cross entropy is equivalent to minimizing KL divergence. This is because cross entropy is equal to (entropy of the true distribution)

In NN training, minimizing cross entropy is equivalent to minimizing KL divergence. This is because cross entropy is equal to (entropy of the true distribution) + (KL divergence from the true distribution to the model). Obviously by changing the model you can't change the first term, only the second. So when you minimize cross entropy, you are minimizing KL divergence. Minimizing mean-squared-error loss is equivalent to minimizing KL divergence (and thus cross entropy) under the assumption that your model produces a vector that parameterizes the mean of a multivariate Gaussian distribution which is then used to predict your data. This is the most natural way to set up a model that predicts continuous data.