3 ms·
This article seems to imply that the author equates Shannon entropy with information. Is that so? It has always struck me that Shannon entropy is a good measure
by yafujifide 16y ago
This article seems to imply that the author equates Shannon entropy with information. Is that so? It has always struck me that Shannon entropy is a good measure of information, but is not a good definition of information itself. Like the difference between a vector and the magnitude of a vector, information and its measure are not the same. What is information itself? If this author actually has a definition of that, I would like to see it.
- Anon84 16y ago"You should call it entropy, for two reasons. In the first place your uncertainty function has been used in statistical mechanics under that name, so it already has a name. In the second place, and more important, no one really knows what entropy really is, so in a debate you will always have the advantage." John von Neumann to Claude Shannon
- troystribling 16y agoShannon does not define information but the entropy of information.
- jey 16y agoHow about http://en.wikipedia.org/wiki/Kolmogorov_complexity http://en.wikipedia.org/wiki/Kolmogorov_complexity ? The big problem with Shannon entropy is that it's measured with respect to a probability space, so it's not an "absolute" measure of information. Kolmogorov complexity "solves" this problem by defining information content of a string as the length of the shortest program that generates the string. (You still have to pick a reference UTM, but this only affects the program lengths at worst by a constant.) [Disclaimer: Hobbyist, not expert.]