3 ms·
I'm reading that post, and while fascinating, I don't understand the entropy calculation. The digit chance is easy: For a single digit to appear one needs 1/3
by WirelessGigabit 3y ago
I'm reading that post, and while fascinating, I don't understand the entropy calculation.
The digit chance is easy:
For a single digit to appear one needs 1/3 * 1/10 = 1/30.
For a single letter to appear one needs 1/3 * 1/26 = 1/78.
But the bits of entropy throw me off. 26 + 26 + 10 = 62, which is 2^5.954. But that is for a uniform distribution. The writer states that it actually is 2^5.826, or 1/~56.7. I don't get how they to that number.
- lukevalenta 3y agoHere's how you can get that entropy using the Shannon entropy equation from https://en.wikipedia.org/wiki/Entropy_(information_theory) https://en.wikipedia.org/wiki/Entropy_(information_theory). Like you said, the probability of any digit appearing is 1/30, and there are 10 digits. The probability of a lower- or upper- case letter appearing is 1/78, and there are 26+26 = 52 letters. Plugging that into the formula for Shannon entropy, we get this: - 10 * (1/30) * log_2(1/30) - 52 * (1/78) * log_2(1/78) =~ 5.826