2 ms·
But isn't the information gained from a message necessarily subjective? It depends on just on this message, but from the distribution from which this message wa
by ssivark 7y ago
But isn't the information gained from a message necessarily subjective? It depends on just on this message, but from the distribution from which this message was drawn, and more appropriately, the distribution anticipated by the observer will determine the value (information content) of the message. That is exactly how it becomes useful in practice.
This "surprisal" can be parametrized by relative entropy (Kullback-Liebler divergence), which reduces to the Shannon entropy formula only when the observer has the correct anticipated distribution.
- noema 7y agoI think there is a distinction to be made between information in the ontological sense and information in the pragmatic sense. The former does not have any purchase on "meaning" or "use", only a metric of informational entropy or degrees of freedom, irrespective of observation. However, the sense in which you speak of information is the pragmatic dimension, where an observer receives a datum of semantic content against a given backdrop of pre-established "meaning". This sense is problematic as an objective metric.