3 ms·

Thank you. I just learned about Shannon Entropy and that it grows with information content. The negative sign in the formula is due to the negative log of the n

Thank you. I just learned about Shannon Entropy and that it grows with information content. The negative sign in the formula is due to the negative log of the normalized population probability. https://math.stackexchange.com/questions/395121/how-entropy-scales-with-sample-size https://math.stackexchange.com/questions/395121/how-entropy-...

inimino 3y ago
Yes. You can also take the log of the reciprocal probability (or "luck") and then you don't need the negative sign. This value is the expected number of trials you would need for the outcome to occur once. I find this presentation a bit more intuitive. See also Boltzmann entropy S = k log W which takes this form.