3 ms·
Yes, they wouldn't be normalized so you can't interpret it as a probability. Softmax is the function that normalizes it. However, even that you can't interpret
by cultus 8y ago
Yes, they wouldn't be normalized so you can't interpret it as a probability. Softmax is the function that normalizes it. However, even that you can't interpret as a probability of some outcome, because you do not take into account the prior.
For that, you need actual Bayesian techniques.
- anonymousDan 8y agoCan you elaborate on what you mean by not taking into account the prior? What is the output of aoftmax if not a probability?
- cultus 8y agoWell, it's a probability of the output given the fixed, fitted model parameters. This is the likelihood. This is not the probability of getting that output, since we don't know what the parameters should really be. See the inverse probability fallacy [0]. This confusion is literally the main reason behind the replication crisis of scientific research. In ML, you typically have a model f(w;u), and you are trying to fit the parameters w to some data, given some hyperparameters u that represent some kind of prior information. In frequentist techniques, these could be regularization parameters. In frequentist techniques, the hyperparameters the fitted parameters are assumed fixed, and not random. In contrast, Bayesian methods assume it is random as well. This extra uncertainty is neglected by the predictions of frequentist techniques. Nonetheless, the uncertainty is real. [0] https://en.wikipedia.org/wiki/Confusion_of_the_inverse https://en.wikipedia.org/wiki/Confusion_of_the_inverse
- anonymousDan 8y agoThanks!