4 ms·

(2015)

• amoss 3y ago
  Very enjoyable and well-written story until: > After all, it had been Wiener who discovered a precise mathematical definition of information: The higher the probability, the higher the entropy and the lower the information content. This seems jarring wrong and a little digging reveals that in 1948 Weiner wrote: > the notion of information attaches itself very naturally to a classical notion in statistical mechanics: that of entropy [...] just as the amount of information in a system is a measure of its degree of organization, so the entropy of a system is a measure of its degree of disorganization; and the one is simply the negative of the other. The broad idea is there (and perhaps Weiner deserves credit for the observation), but it is the same year that Shannon published A Mathematical Theory of Communication, and two years later Weiner is quoted with the more precise description: > Just as entropy is a measure of disorganization, the information carried by a set of messages is a measure of organization. In fact, it is possible to interpret the information carried by a message as essentially the negative of its entropy, and the negative logarithm of its probability. The later quote using Shannon's definition. I've not seen information theory being credited to Weiner before and that makes me curious - what is the story here? Parallel invention is commonplace in scientific history, yet Weiner and Shannon are listed as collaborators in other sources.