3 ms·
> encode the difference between what is predicted and the actual data Minor nitpick: In the idealized model there is no single prediction that you can take the
by remcob 7y ago
> encode the difference between what is predicted and the actual data
Minor nitpick: In the idealized model there is no single prediction that you can take the difference with. There is just a probability distribution and you encode the actual data using this distribution.
Taking the difference between the most likely prediction and the actual data is just a very common implementation strategy.
- anamexis 7y agoHow is "the most likely prediction" not a "single prediction that you can take the difference with"?
- derf_ 7y agoWhat if there are two (nearly) equally likely predictions? What if there are N, and they are not close together? "A single prediction that you can take the difference with" makes some assumptions about the shape of your distribution, at least under any reasonable model of "code the difference" where, e.g., probability decreases as the difference gets larger. These are often very good assumptions, to be fair.