Fascinating discussion. I see some parallel here to the Bayesian vs Frequentist view of probability. They are perhaps both valid points of view depending on th

Fascinating discussion. I see some parallel here to the Bayesian vs Frequentist view of probability. They are perhaps both valid points of view depending on the situation. If you take a frequentist view of an unbiased coin, then the probability that it will land heads on the next flip is objectively, by definition 50%. So the resulting calculation of entropy (log 50% = 1 bit) is also objectively defined. But if your 50% probability represents a subjective belief, the resulting entropy calculation should also be considered subjective, I would think.