4 ms·

There seems to be a great, unresolved difference of opinion about whether information entropy and thermodynamic entropy are commensurable. In trying to connect

There seems to be a great, unresolved difference of opinion about whether information entropy and thermodynamic entropy are commensurable. In trying to connect Shannon entropy to thermodynamic entropy, I always get stuck on the fact that you need to have a defined alphabet or symbol set.

Q_is_4_Quantum 7y ago
It seemed clear to me at the time when I read (a few of - but especially https://bayes.wustl.edu/etj/articles/theory.1.pdf https://bayes.wustl.edu/etj/articles/theory.1.pdf) ET Jaynes' papers on classical thermodynamics that they were commensurable. I'm mildly surprised there is some question about that!
wrnr 7y ago
vs an arbitrary defined number of dimensions? Their are more exotic mechanisms to define entropy: https://en.wikipedia.org/wiki/Parastatistics https://en.wikipedia.org/wiki/Parastatistics
- dr_dshiv 7y ago
  In a thermodynamic sense, information input (say, of photons) is generally going to increase the entropy of a receiving system. Only certain kinds of information can reduce the entropy of the receiving system. Too much information (i.e., too many photons) can literally burn your eyes. I don't see how to square those manifest physical effects with the immateriality of Shannon information entropy.
  - wrnr 7y ago
    Is this not the problem of Maxwell's Daemon, if an intelligent agent observes and interferes with a particle system to reduce the system's entropy then overall entropy is still increasing because the agents needs to store and calculate information in some physical medium, be it brain matter of silicon.
    - dr_dshiv 7y ago
      Not exactly, I don't think. In the Shannon formulation, there isn't a notion of "too much information" -- it is all symbols. But in reality, too many photons can burn our eyes or a denial of service attack can break a website. Too much information is a reality --because we are thermodynamic entities. I don't see how to understand that with Shannon.
      - wrnr 7y ago
        Yeah right, and too much heat can burn my toast, forget about global warming it about universal warming, I don't understand that with Einstein either.
hackinthebochs 7y ago
The alphabet is to distinguish states. But this is the case in thermodynamic entropy as well, that each state is distinguishable. What is important is quantifying the set of possible states.

Q_is_4_Quantum 7y ago

It seemed clear to me at the time when I read (a few of - but especially https://bayes.wustl.edu/etj/articles/theory.1.pdf https://bayes.wustl.edu/etj/articles/theory.1.pdf) ET Jaynes' papers on classical thermodynamics that they were commensurable. I'm mildly surprised there is some question about that!

wrnr 7y ago

vs an arbitrary defined number of dimensions? Their are more exotic mechanisms to define entropy: https://en.wikipedia.org/wiki/Parastatistics https://en.wikipedia.org/wiki/Parastatistics

dr_dshiv 7y ago
In a thermodynamic sense, information input (say, of photons) is generally going to increase the entropy of a receiving system. Only certain kinds of information can reduce the entropy of the receiving system. Too much information (i.e., too many photons) can literally burn your eyes. I don't see how to square those manifest physical effects with the immateriality of Shannon information entropy.
- wrnr 7y ago
  Is this not the problem of Maxwell's Daemon, if an intelligent agent observes and interferes with a particle system to reduce the system's entropy then overall entropy is still increasing because the agents needs to store and calculate information in some physical medium, be it brain matter of silicon.
  - dr_dshiv 7y ago
    Not exactly, I don't think. In the Shannon formulation, there isn't a notion of "too much information" -- it is all symbols. But in reality, too many photons can burn our eyes or a denial of service attack can break a website. Too much information is a reality --because we are thermodynamic entities. I don't see how to understand that with Shannon.
    - wrnr 7y ago
      Yeah right, and too much heat can burn my toast, forget about global warming it about universal warming, I don't understand that with Einstein either.

hackinthebochs 7y ago

The alphabet is to distinguish states. But this is the case in thermodynamic entropy as well, that each state is distinguishable. What is important is quantifying the set of possible states.