I'd highly recommend Arieh Ben-Naim's books on entropy for anyone looking for more on this point. TLDR he makes the argument that entropy is a measure of the p

I'd highly recommend Arieh Ben-Naim's books on entropy for anyone looking for more on this point. TLDR he makes the argument that entropy is a measure of the probability of probability distributions.