3 ms·
Shannon entropy works for anything you can assign an alphabet of symbols to. It doesn't need the thing itself to be symbolic, just the description of the thing.
by SAI_Peregrinus 5y ago
Shannon entropy works for anything you can assign an alphabet of symbols to. It doesn't need the thing itself to be symbolic, just the description of the thing.
- dr_dshiv 5y agoThat seems like a strong claim. Any proof?
- SAI_Peregrinus 5y agoSure. https://people.math.harvard.edu/~ctm/home/text/others/shannon/entropy/entropy.pdf https://people.math.harvard.edu/~ctm/home/text/others/shanno... Particularly part 4, The Continuous Channel. No discrete symbols there!
- dr_dshiv 5y agoMaybe I’m not reading it properly, but in part 5, where he deals with transmission of continuous signals he doesn’t use his entropy formulation. He uses a distance metric, showing the difference between the sent signal and the received.