Can you explain this in more detail? Entropy is calculated as a function of a probability distribution over possible messages or symbols. The sender might have

Can you explain this in more detail? Entropy is calculated as a function of a probability distribution over possible messages or symbols. The sender might have a distribution P over possible symbols, and the receiver might have another distribution Q over possible symbols. Then the "true" distribution over possible symbols might be another distribution yet, call it R. The mismatch between these is what leads to various inefficiencies in coding, decoding, etc [1]. But both P and Q are beliefs about R -- that is, they are properties of observers. [1] https://en.wikipedia.org/wiki/Kullback–Leibler_divergence#Coding https://en.wikipedia.org/wiki/Kullback–Leibler_divergence#Co...