4 ms·
If you look deep enough they are not exclusive. From the deeplearning book: "Many authors use the term “cross-entropy” to identify specifically the negative log
by imjonse 3y ago
If you look deep enough they are not exclusive. From the deeplearning book:
"Many authors use the term “cross-entropy” to identify specifically the negative log-likelihood of a Bernoulli or softmax distribution, but that is a misnomer. Any loss consisting of a negative log-likelihood is a cross-
entropy between the empirical distribution defined by the training set and the
probability distribution defined by model. For example, mean squared error is the
cross-entropy between the empirical distribution and a Gaussian model"
https://stats.stackexchange.com/questions/288451/why-is-mean-squared-error-the-cross-entropy-between-the-empirical-distribution-a https://stats.stackexchange.com/questions/288451/why-is-mean...
- yobbo 3y agoAlso, the gradients of "softmax loss" and mean-square-error loss are the same. The network learns to optimize the same function, up until the output activation.