4 ms·
> If you have two independent events it makes sense that the total surprisal (information content) should be the sum of the surprisals of the two events. I thi
by ramblenode 3y ago
> If you have two independent events it makes sense that the total surprisal (information content) should be the sum of the surprisals of the two events.
I think the parent is also asking why we would expect surprisal to be additive rather than multiplicative like probabilities.
- Majromax 3y agoBecause if two things happen – one totally expected and one very surprising – then on net you’re still surprised.
- ramblenode 3y agoThat's also the case for probabilities.
- joelthelion 3y agoI think in part because it's really nice to measure information in bits rather than tiny probabilities. And it aligns well with how information is stored.