It does indeed sound like it’s written the wrong way around. Random information = high entropy. A common word is therefore low entropy. Here’s a good analogy

It does indeed sound like it’s written the wrong way around. Random information = high entropy. A common word is therefore low entropy. Here’s a good analogy I found: > Informally, the amount of information in an email is proportional to the amount of “surprise” its reading causes. For example, if an email is simply a repeat of an earlier email, then it is not informative at all. On the other hand, if say the email reveals the outcome of a cliff-hanger election, then it is highly informative. Similarly, the information in a variable is tied to the amount of surprise that value of the variable causes when revealed. Shannon’s entropy quantifies the amount of information in a variable, thus providing the foundation for a theory around the notion of information. https://arxiv.org/pdf/1405.2061.pdf https://arxiv.org/pdf/1405.2061.pdf