Why is it an integer number? Entropy (in bits) is defined as - \sum_x (p(x) log_2 p(x)) There is no reason this has to be an integer, since probabilities are

Why is it an integer number? Entropy (in bits) is defined as - \sum_x (p(x) log_2 p(x)) There is no reason this has to be an integer, since probabilities are not restricted to being reciprocals of powers of 2. Consider also that you can simply use a different logarithm base to get a different unit (e.g. use the natural logarithm to obtain the entropy in nats). It would be bizarre if the arbitrary choice of 2 as the base gave a unit that was indivisible. I think this whole confusion comes down to the difference between a bit as a "unit of information in the sense of information theory" [divisible] and a bit as a "single physical one or zero" [not divisible]. The relationship between the two is that the entropy of a random variable is a lower bound on the average number of bits required to represent it.