5 ms·

> It is not mundane, and it is also not right, at least for entropy in Physics and Thermodynamics. Articles about MaxEnt thermodynamics by E.T. Jaynes where he

> It is not mundane, and it is also not right, at least for entropy in Physics and Thermodynamics. Articles about MaxEnt thermodynamics by E.T. Jaynes where he talks about the “anthropomorphic” nature of entropy date back to 1960s. How is that not right in physics?

kergonath 2y ago
By the look of it, it is another misguided attempt to apply information theory concepts to thermodynamics. Entropy as information is seductive because that way we think we can understand it better, and it looks like it works. But we need to be careful because even though we can get useful insights from it (like Hawking radiation) it’s easy to reach unphysical conclusions. > How is that not right in physics? Why would it be right? Was it used to make predictions that were subsequently verified?
- kgwgk 2y ago
  That misguided attempt has resulted in full graduate courses and textbooks. Maybe one look is not all that it takes to fully assess its worthiness. https://www.amazon.com/Microphysics-Macrophysics-Applications-Statistical-Mathematical/dp/3540454691 https://www.amazon.com/Microphysics-Macrophysics-Application... https://arxiv.org/pdf/cond-mat/0501322 https://arxiv.org/pdf/cond-mat/0501322
  - kergonath 2y ago
    Plenty of bad stuff made its way into textbooks, I saw some really dodgy stuff at uni. And for every uncontroversial statement we can find a textbook that argues that it is wrong. Sorting the good from the bad is the main point of studying science and it is not easy. What is also important is that approaches that can work in some field or context might be misleading or lead to wrong outcomes in others. Information theory is obviously successful and there is nothing fundamentally wrong with it. Where we should be careful is when we want to apply some reasoning verbatim to a different problem. Sometimes it works, and sometimes it does not. Entropy is a particularly good example. It is abstract enough to be mysterious for a vast majority of the population, hence why these terribly misleading vulgarisation articles pop up so often. Thinking of it in terms of information is sometimes useful, but going from information to knowledge is a leap, and then circling back to Physics is a bit adventurous.
    - kgwgk 2y ago
      Plenty of bad stuff makes its way into hackernews comments as well. Saying that “it could be wrong” doesn’t really support the “it’s wrong” claim, does it?
      - kergonath 2y ago
        I did not make that point, though. I rejected an appeal to authority because something was in a textbook. I made no comment about the validity of that person’s work in his field, I just pointed out that this transferability was limited.
        kgwgk 2y ago
        My bad, I thought you considered Balian’s work another misguided attempt to apply information theory concepts to thermodynamics. For the record, this is the abstract of the “Information in statistical physics” article: “We review with a tutorial scope the information theory foundations of quantum statistical physics. Only a small proportion of the variables that characterize a system at the microscopic scale can be controlled, for both practical and theoretical reasons, and a probabilistic description involving the observers is required. The criterion of maximum von Neumann entropy is then used for making reasonable inferences. It means that no spurious information is introduced besides the known data. Its outcomes can be given a direct justification based on the principle of indifference of Laplace. We introduce the concept of relevant entropy associated with some set of relevant variables; it characterizes the information that is missing at the microscopic level when only these variables are known. For equilibrium problems, the relevant variables are the conserved ones, and the Second Law is recovered as a second step of the inference process. For non-equilibrium problems, the increase of the relevant entropy expresses an irretrievable loss of information from the relevant variables towards the irrelevant ones. Two examples illustrate the flexibility of the choice of relevant variables and the multiplicity of the associated entropies: the thermodynamic entropy (satisfying the Clausius–Duhem inequality) and the Boltzmann entropy (satisfying the H-theorem). The identification of entropy with missing information is also supported by the paradox of Maxwell’s demon. Spin-echo experiments show that irreversibility itself is not an absolute concept: use of hidden information may overcome the arrow of time.”
- oh_my_goodness 2y ago
  "We need to be careful", agreed. "It's not mundane", agreed. (It's mundane in information theory because that's how they define entropy.) "It's [...] not right" (from your first comment), can you give/link a specific physical example? It would be very cool to have a clear counterexample.
  - kergonath 2y ago
    > can you give/link a specific physical example? About the lack of subjectivity of the states? If we consider any bit of matter (for example a crystal or an ideal gas), the macrostate is completely independent of the observer: it’s just the state in which the law of physics say that bit of matter should be. In an ideal gas it is entirely determined by the pressure and volume, which are anything but subjective. For a crystal it is more complex because we have to account for things like its shape but the reasoning is the same. Then, the microstates are just accessible states, and this is also dictated by Physics. For example, it is quite easy to see that a crystal has fewer accessible states than a gas (the atoms’ positions are constrained and the velocities are limited to the crystal’s vibration modes). We can calculate the entropy in the experimental conditions within that framework, or in the case of correlated liquids, or amorphous solids, or whatever. But the fact that we can come up with different entropies if we make different hypotheses does not mean that any of these hypotheses is actually valid. If we measure the entropy directly we might have a value that is consistent with several models, or none. The actual entropy is what we observe, not the theoretical scaffolding we use to try to make sense of it. And again, this is not subjective.
    - oh_my_goodness 2y ago
      Agreed, sure. Of course it's not subjective. Is there a concrete physical example where the information-theory definition of entropy conflicts with experiment?
      - tsimionescu 2y ago
        Others in this thread do believe that entropy is a subjective measure, or more precisely a measurement of the information that an observer has about a system instead of a measurement about the state of the system itself. Information theory easily leads to this interpretation, since for example the informational content of a stream of bytes can very much be observer-dependent. For example, a perfectly encrypted stream of all 1s will appear to have very high entropy for anyone who doesn't know the decryption process and key, while in some sense it will be interpreted as a stream with entropy 0 by someone who knows the decryption process and key. Of course, the example I gave is arguable, since the two observers are not actually observing the same process. One is looking at enc(x), the other is looking at x. They would both agree that enc(x) has high entropy, and x has low entropy. But this same kind of phenomenon doesn't work with physical entropy. A gas is going to burn my hand or not regardless of how well I know its microstates.

kergonath 2y ago

By the look of it, it is another misguided attempt to apply information theory concepts to thermodynamics. Entropy as information is seductive because that way we think we can understand it better, and it looks like it works. But we need to be careful because even though we can get useful insights from it (like Hawking radiation) it’s easy to reach unphysical conclusions. > How is that not right in physics? Why would it be right? Was it used to make predictions that were subsequently verified?

kgwgk 2y ago
That misguided attempt has resulted in full graduate courses and textbooks. Maybe one look is not all that it takes to fully assess its worthiness. https://www.amazon.com/Microphysics-Macrophysics-Applications-Statistical-Mathematical/dp/3540454691 https://www.amazon.com/Microphysics-Macrophysics-Application... https://arxiv.org/pdf/cond-mat/0501322 https://arxiv.org/pdf/cond-mat/0501322
- kergonath 2y ago
  Plenty of bad stuff made its way into textbooks, I saw some really dodgy stuff at uni. And for every uncontroversial statement we can find a textbook that argues that it is wrong. Sorting the good from the bad is the main point of studying science and it is not easy. What is also important is that approaches that can work in some field or context might be misleading or lead to wrong outcomes in others. Information theory is obviously successful and there is nothing fundamentally wrong with it. Where we should be careful is when we want to apply some reasoning verbatim to a different problem. Sometimes it works, and sometimes it does not. Entropy is a particularly good example. It is abstract enough to be mysterious for a vast majority of the population, hence why these terribly misleading vulgarisation articles pop up so often. Thinking of it in terms of information is sometimes useful, but going from information to knowledge is a leap, and then circling back to Physics is a bit adventurous.
  - kgwgk 2y ago
    Plenty of bad stuff makes its way into hackernews comments as well. Saying that “it could be wrong” doesn’t really support the “it’s wrong” claim, does it?
    - kergonath 2y ago
      I did not make that point, though. I rejected an appeal to authority because something was in a textbook. I made no comment about the validity of that person’s work in his field, I just pointed out that this transferability was limited.
      - kgwgk 2y ago
        My bad, I thought you considered Balian’s work another misguided attempt to apply information theory concepts to thermodynamics. For the record, this is the abstract of the “Information in statistical physics” article: “We review with a tutorial scope the information theory foundations of quantum statistical physics. Only a small proportion of the variables that characterize a system at the microscopic scale can be controlled, for both practical and theoretical reasons, and a probabilistic description involving the observers is required. The criterion of maximum von Neumann entropy is then used for making reasonable inferences. It means that no spurious information is introduced besides the known data. Its outcomes can be given a direct justification based on the principle of indifference of Laplace. We introduce the concept of relevant entropy associated with some set of relevant variables; it characterizes the information that is missing at the microscopic level when only these variables are known. For equilibrium problems, the relevant variables are the conserved ones, and the Second Law is recovered as a second step of the inference process. For non-equilibrium problems, the increase of the relevant entropy expresses an irretrievable loss of information from the relevant variables towards the irrelevant ones. Two examples illustrate the flexibility of the choice of relevant variables and the multiplicity of the associated entropies: the thermodynamic entropy (satisfying the Clausius–Duhem inequality) and the Boltzmann entropy (satisfying the H-theorem). The identification of entropy with missing information is also supported by the paradox of Maxwell’s demon. Spin-echo experiments show that irreversibility itself is not an absolute concept: use of hidden information may overcome the arrow of time.”
oh_my_goodness 2y ago
"We need to be careful", agreed. "It's not mundane", agreed. (It's mundane in information theory because that's how they define entropy.) "It's [...] not right" (from your first comment), can you give/link a specific physical example? It would be very cool to have a clear counterexample.
- kergonath 2y ago
  > can you give/link a specific physical example? About the lack of subjectivity of the states? If we consider any bit of matter (for example a crystal or an ideal gas), the macrostate is completely independent of the observer: it’s just the state in which the law of physics say that bit of matter should be. In an ideal gas it is entirely determined by the pressure and volume, which are anything but subjective. For a crystal it is more complex because we have to account for things like its shape but the reasoning is the same. Then, the microstates are just accessible states, and this is also dictated by Physics. For example, it is quite easy to see that a crystal has fewer accessible states than a gas (the atoms’ positions are constrained and the velocities are limited to the crystal’s vibration modes). We can calculate the entropy in the experimental conditions within that framework, or in the case of correlated liquids, or amorphous solids, or whatever. But the fact that we can come up with different entropies if we make different hypotheses does not mean that any of these hypotheses is actually valid. If we measure the entropy directly we might have a value that is consistent with several models, or none. The actual entropy is what we observe, not the theoretical scaffolding we use to try to make sense of it. And again, this is not subjective.
  - oh_my_goodness 2y ago
    Agreed, sure. Of course it's not subjective. Is there a concrete physical example where the information-theory definition of entropy conflicts with experiment?
    - tsimionescu 2y ago
      Others in this thread do believe that entropy is a subjective measure, or more precisely a measurement of the information that an observer has about a system instead of a measurement about the state of the system itself. Information theory easily leads to this interpretation, since for example the informational content of a stream of bytes can very much be observer-dependent. For example, a perfectly encrypted stream of all 1s will appear to have very high entropy for anyone who doesn't know the decryption process and key, while in some sense it will be interpreted as a stream with entropy 0 by someone who knows the decryption process and key. Of course, the example I gave is arguable, since the two observers are not actually observing the same process. One is looking at enc(x), the other is looking at x. They would both agree that enc(x) has high entropy, and x has low entropy. But this same kind of phenomenon doesn't work with physical entropy. A gas is going to burn my hand or not regardless of how well I know its microstates.