7 ms·

Entropy (differences) are an objective quantity which can be measured, there is no subjectivity about it. It is not which parameters you know it is about which

Entropy (differences) are an objective quantity which can be measured, there is no subjectivity about it. It is not which parameters you know it is about which parameters you hold fixed.

oldsecondhand 4y ago
I think he is confusing the usage of entropy in physics and computer science. In computer science entropy is conditional probability and depends on what we know about a system.
- kgwgk 4y ago
  As it does in physics! "which parameters [thermodynamic variables] you know" ~ "which parameters [thermodynamic variables] you hold fixed" (or know in average, like the energy for a system in a heat bath where the temperature is fixed) https://bayes.wustl.edu/etj/articles/theory.1.pdf https://bayes.wustl.edu/etj/articles/theory.1.pdf http://nicf.net/articles/thermodynamics-statistical-mechanics/ http://nicf.net/articles/thermodynamics-statistical-mechanic...
  - oldsecondhand 4y ago
    You can observe the movement of molecules beyond macro properties like temperature.
    - kgwgk 4y ago
      Sure, at least in principle. And if you knew what every molecule was doing the entropy would vanish.
      - IX-103 4y ago
        I've thought that for a while, but I'm not a physicist. Do you know any prominent physicists that hold that view? It seems to contradict at least the popular narrative about entropy as a property of a system.
        kgwgk 4y ago
        > Do you know any prominent physicists that hold that view? The view that the entropy of a microstate (i.e. a perfectly defined physical state) is zero? All of them, hopefully.
        liminvorous 4y ago
        I only have an undergraduate degree in physics, but I think the point you may be missing is that information requires some physical medium to store it. So entropy is very roughly speaking the property of the system that determines how big a hard drive you need to store a description of that system.
toxik 4y ago
Leonard Susskind disagrees with you. See his lectures on statistical mechanics, he is very clear that entropy is a matter of knowledge about the system. It has to be.
- oh_my_goodness 4y ago
  Susskind says that entropy is determined by selecting a macro-state. He doesn't claim that the entropy of a macro-state depends on whether we know which macro-state the real system is really in. If we happen to know, then, sure. For example we could pick a weird-ass observable state, and when we saw it we would know the entropy of the system was low. But the entropy of each macro-state just depends on how many micro-states we define it to contain. It doesn't depend on our knowledge of the system state.
  - jbay808 4y ago
    The concept of entropy wasn't invented so that we could calculate entropies of macrostates, it was so that we could calculate entropies of real systems and understand their behaviour. Macrostates are an accounting tool that helps us do this. You seem to be treating the calculation of macrostate entropy as an end-goal in itself, but also allowing yourself to somehow freely choose any macrostate you want. When it comes to applying thermodynamics in practice, you'll have to calculate the entropy of a real, or at least hypothetical, system. The point of macrostates is that you ought to know which macrostate a given system is in. That's the thing that you know. You don't know which microstate it's in, but you do know which macrostate it's in. For example, if I say "a cubic metre volume of air at room temperature and pressure", I've described a physical system. I've also described a macrostate. If you're calculating the entropy of macrostates that are not consistent with a description of a system -- if you've defined your macrostates such that you don't know which macrostate a given system is in -- then in order to calculate that system's entropy you have to sum up over all such possible macrostates anyway, so you haven't saved yourself any work or earned any insights along the way. So yes, you can calculate the entropy of a macrostate without knowing what macrostate a real system is in, but it kind of sounds like you're arguing that log(x) is not a function of variable x, because log(3) is a constant and log(4) is a constant, and you can divide up any x into constants of your choice.
jbay808 4y ago
No, it is subjective. We just only have such blunt instruments for practically measuring states, relative to the gargantuan amount of entropy in most real systems, that the subjective nature of entropy is easy to miss. But in a world where the frontiers of thermodynamics have moved from steam engines to lasers, computers, DNA, and black holes, the difference is increasingly obvious and important. With steam engines, we got away with treating a volume of gas as having not only a few parameters that we knew and cared about, like mass, temperature and pressure, but we could further deceive ourselves into thinking that those were the only parameters that existed to describe the system. The only parameters that were knowable. But Boltzmann knew better. Look at Boltzmann's formula, S = kB log W. For any single particular system you describe to me, W will be 1, and so S will be 0. So it's only if you describe an ensemble of systems -- that is, if you describe a system vaguely, such that I am left to imagine the details -- that we have nonzero entropy. If you ask me to calculate the entropy of that "system", that macrostate, that ensemble, then sure, I'll end up with nonzero entropy. But if I ask you to keep transmitting more data about the scenario, then with each further description, you'll be narrowing the state space and thereby decreasing the entropy. Look, since the entropy of a macrostate is nonzero, but the entropy of any single microstate which is consistent with that macrostate is zero, it's clear that entropy is not an intrinsic property of any real system. It's a property of how many other possible non-existent systems could be swapped out for the one in front of you, without you noticing the change. If I swap out the air in your room for an equal volume of air at equal temperature and pressure, you probably won't notice. If I swap out the hard drive in your laptop for an equal volume of hard drive at equal temperature and pressure, you probably will!
- deltaonefour 4y ago
  Very few people know this but. Information entropy and statistical mechanical entropy are two different things. They share the same equation and the same name but they are two unrelated concepts. You have conflated the two. The person you are responding to is referring to statistical entropy. Basically in this entire thread nobody, including you, is fully grasping the situation.
  - jbay808 4y ago
im3w1l 4y ago
I've been trying to reconcile these perspectives, and I think it really is both. And they are both physically relevant. Consider the subjective entropy perspective. If you know the exact microstate of a system, then you can in theory play the part of Maxwell's demon. You could have a little gate that you open only for fast particles, and using your knowledge of the microstate, you can predict exactly when they will arrive. But consider the objective perspective. If you take this very same system and put it in thermal contact with another system, then an objective entropy perspective is the relevant one. Those systems will equilibrize and your subjective knowledge is irrelevant to that process. I haven't fully wrapped my head around it yet, but I do think that acknowledging both is a step in the right direction at least.
- jbay808 4y ago
  > If you take this very same system and put it in thermal contact with another system, then an objective entropy perspective is the relevant one. Those systems will equilibrize and your subjective knowledge is irrelevant to that process. The subjective view handles this scenario just fine, though, and makes more accurate predictions than the objective view. For example, there are systems where some aspects of the original microstate survive thermal contact with another system. We use such systems to store data! I bet your hard drive is in thermal contact with its environment right now! It's very hard to reconcile this with an objective take on entropy. And there are some systems that will rapidly be scrambled. The subjective perspective has no problem admitting that your knowledge of a system can become inaccurate and useless. Even without thermal contact, you'd need to perform a tremendous amount of (perhaps reversible) computation in order to make a functioning Maxwell's demon with your initial microstate conditions, because the microstate will evolve in time in a complicated way. The subjective view is still totally consistent with entropy of a system increasing over time!
  - deleted 4y ago
    [deleted]
  - deleted 4y ago
    [deleted]
  - 4y ago
jwarden 4y ago
Fascinating discussion. I see some parallel here to the Bayesian vs Frequentist view of probability. They are perhaps both valid points of view depending on the situation. If you take a frequentist view of an unbiased coin, then the probability that it will land heads on the next flip is objectively, by definition 50%. So the resulting calculation of entropy (log 50% = 1 bit) is also objectively defined. But if your 50% probability represents a subjective belief, the resulting entropy calculation should also be considered subjective, I would think.
alecst 4y ago
U, the internal energy is objective. The free energy F = U - TS is the maximum amount of work you can extract from the system. This depends on how much you know about the system. S does indeed depend on what you know about the system. See the Gibbs Paradox for more information.
- mannykannot 4y ago
  If two people disagree on the maximum amount of work that could be extracted from a given system (with both of them basing their figure on their own evaluation of S), are there any cases where it would it be impossible to empirically demonstrate that at least the proponent of the lower figure was wrong? If only changes in S (and F) have measurable consequences, would that not merely mean that assigning an absolute value is an arbitrary choice, which would not mean the same as it being subjective (there could still be an objective conversion between one basis and another, as there is for kinetic energy in different inertial reference frames.) In the Gibbs Paradox, there is no subjectivity in whether the gases being mixed are the same or different, and no subjectivity in what the change of entropy is in either case. The paradox is that it does not feel right that identity makes an objective difference between the two cases, but the empirically-demonstrable distinction between fermions and bosons shows that this intuition does not hold in general. I believe Von Neumann came up with a QM resolution of the paradox.
  - jbay808 4y ago
    > are there any cases where it would it be impossible to empirically demonstrate that at least the proponent of the lower figure was wrong? Isn't it more interesting to examine a situation where it would be possible to empirically demonstrate that the proponent of the lower figure was wrong?
    - mannykannot 4y ago
      In that case we could objectively say that one value of S does not yield F for that system (given that F is defined as a maximum), but this would not resolve the general question of subjectivity.

Entropy (differences) are an objective quantity which can be measured, there is no subjectivity about it. It is not which parameters you know it is about which parameters you hold fixed.

oldsecondhand 4y ago

I think he is confusing the usage of entropy in physics and computer science. In computer science entropy is conditional probability and depends on what we know about a system.

kgwgk 4y ago
As it does in physics! "which parameters [thermodynamic variables] you know" ~ "which parameters [thermodynamic variables] you hold fixed" (or know in average, like the energy for a system in a heat bath where the temperature is fixed) https://bayes.wustl.edu/etj/articles/theory.1.pdf https://bayes.wustl.edu/etj/articles/theory.1.pdf http://nicf.net/articles/thermodynamics-statistical-mechanics/ http://nicf.net/articles/thermodynamics-statistical-mechanic...
- oldsecondhand 4y ago
  You can observe the movement of molecules beyond macro properties like temperature.
  - kgwgk 4y ago
    Sure, at least in principle. And if you knew what every molecule was doing the entropy would vanish.
    - IX-103 4y ago
      I've thought that for a while, but I'm not a physicist. Do you know any prominent physicists that hold that view? It seems to contradict at least the popular narrative about entropy as a property of a system.
      - kgwgk 4y ago
        > Do you know any prominent physicists that hold that view? The view that the entropy of a microstate (i.e. a perfectly defined physical state) is zero? All of them, hopefully.
      - liminvorous 4y ago
        I only have an undergraduate degree in physics, but I think the point you may be missing is that information requires some physical medium to store it. So entropy is very roughly speaking the property of the system that determines how big a hard drive you need to store a description of that system.

toxik 4y ago

Leonard Susskind disagrees with you. See his lectures on statistical mechanics, he is very clear that entropy is a matter of knowledge about the system. It has to be.

oh_my_goodness 4y ago
Susskind says that entropy is determined by selecting a macro-state. He doesn't claim that the entropy of a macro-state depends on whether we know which macro-state the real system is really in. If we happen to know, then, sure. For example we could pick a weird-ass observable state, and when we saw it we would know the entropy of the system was low. But the entropy of each macro-state just depends on how many micro-states we define it to contain. It doesn't depend on our knowledge of the system state.
- jbay808 4y ago
  The concept of entropy wasn't invented so that we could calculate entropies of macrostates, it was so that we could calculate entropies of real systems and understand their behaviour. Macrostates are an accounting tool that helps us do this. You seem to be treating the calculation of macrostate entropy as an end-goal in itself, but also allowing yourself to somehow freely choose any macrostate you want. When it comes to applying thermodynamics in practice, you'll have to calculate the entropy of a real, or at least hypothetical, system. The point of macrostates is that you ought to know which macrostate a given system is in. That's the thing that you know. You don't know which microstate it's in, but you do know which macrostate it's in. For example, if I say "a cubic metre volume of air at room temperature and pressure", I've described a physical system. I've also described a macrostate. If you're calculating the entropy of macrostates that are not consistent with a description of a system -- if you've defined your macrostates such that you don't know which macrostate a given system is in -- then in order to calculate that system's entropy you have to sum up over all such possible macrostates anyway, so you haven't saved yourself any work or earned any insights along the way. So yes, you can calculate the entropy of a macrostate without knowing what macrostate a real system is in, but it kind of sounds like you're arguing that log(x) is not a function of variable x, because log(3) is a constant and log(4) is a constant, and you can divide up any x into constants of your choice.

jbay808 4y ago

No, it is subjective. We just only have such blunt instruments for practically measuring states, relative to the gargantuan amount of entropy in most real systems, that the subjective nature of entropy is easy to miss. But in a world where the frontiers of thermodynamics have moved from steam engines to lasers, computers, DNA, and black holes, the difference is increasingly obvious and important. With steam engines, we got away with treating a volume of gas as having not only a few parameters that we knew and cared about, like mass, temperature and pressure, but we could further deceive ourselves into thinking that those were the only parameters that existed to describe the system. The only parameters that were knowable. But Boltzmann knew better. Look at Boltzmann's formula, S = kB log W. For any single particular system you describe to me, W will be 1, and so S will be 0. So it's only if you describe an ensemble of systems -- that is, if you describe a system vaguely, such that I am left to imagine the details -- that we have nonzero entropy. If you ask me to calculate the entropy of that "system", that macrostate, that ensemble, then sure, I'll end up with nonzero entropy. But if I ask you to keep transmitting more data about the scenario, then with each further description, you'll be narrowing the state space and thereby decreasing the entropy. Look, since the entropy of a macrostate is nonzero, but the entropy of any single microstate which is consistent with that macrostate is zero, it's clear that entropy is not an intrinsic property of any real system. It's a property of how many other possible non-existent systems could be swapped out for the one in front of you, without you noticing the change. If I swap out the air in your room for an equal volume of air at equal temperature and pressure, you probably won't notice. If I swap out the hard drive in your laptop for an equal volume of hard drive at equal temperature and pressure, you probably will!

deltaonefour 4y ago
Very few people know this but. Information entropy and statistical mechanical entropy are two different things. They share the same equation and the same name but they are two unrelated concepts. You have conflated the two. The person you are responding to is referring to statistical entropy. Basically in this entire thread nobody, including you, is fully grasping the situation.
- jbay808 4y ago

im3w1l 4y ago

I've been trying to reconcile these perspectives, and I think it really is both. And they are both physically relevant. Consider the subjective entropy perspective. If you know the exact microstate of a system, then you can in theory play the part of Maxwell's demon. You could have a little gate that you open only for fast particles, and using your knowledge of the microstate, you can predict exactly when they will arrive. But consider the objective perspective. If you take this very same system and put it in thermal contact with another system, then an objective entropy perspective is the relevant one. Those systems will equilibrize and your subjective knowledge is irrelevant to that process. I haven't fully wrapped my head around it yet, but I do think that acknowledging both is a step in the right direction at least.

jbay808 4y ago
> If you take this very same system and put it in thermal contact with another system, then an objective entropy perspective is the relevant one. Those systems will equilibrize and your subjective knowledge is irrelevant to that process. The subjective view handles this scenario just fine, though, and makes more accurate predictions than the objective view. For example, there are systems where some aspects of the original microstate survive thermal contact with another system. We use such systems to store data! I bet your hard drive is in thermal contact with its environment right now! It's very hard to reconcile this with an objective take on entropy. And there are some systems that will rapidly be scrambled. The subjective perspective has no problem admitting that your knowledge of a system can become inaccurate and useless. Even without thermal contact, you'd need to perform a tremendous amount of (perhaps reversible) computation in order to make a functioning Maxwell's demon with your initial microstate conditions, because the microstate will evolve in time in a complicated way. The subjective view is still totally consistent with entropy of a system increasing over time!
- deleted 4y ago
  [deleted]
- deleted 4y ago
  [deleted]
- 4y ago

jwarden 4y ago

Fascinating discussion. I see some parallel here to the Bayesian vs Frequentist view of probability. They are perhaps both valid points of view depending on the situation. If you take a frequentist view of an unbiased coin, then the probability that it will land heads on the next flip is objectively, by definition 50%. So the resulting calculation of entropy (log 50% = 1 bit) is also objectively defined. But if your 50% probability represents a subjective belief, the resulting entropy calculation should also be considered subjective, I would think.

alecst 4y ago

U, the internal energy is objective. The free energy F = U - TS is the maximum amount of work you can extract from the system. This depends on how much you know about the system. S does indeed depend on what you know about the system. See the Gibbs Paradox for more information.

mannykannot 4y ago
If two people disagree on the maximum amount of work that could be extracted from a given system (with both of them basing their figure on their own evaluation of S), are there any cases where it would it be impossible to empirically demonstrate that at least the proponent of the lower figure was wrong? If only changes in S (and F) have measurable consequences, would that not merely mean that assigning an absolute value is an arbitrary choice, which would not mean the same as it being subjective (there could still be an objective conversion between one basis and another, as there is for kinetic energy in different inertial reference frames.) In the Gibbs Paradox, there is no subjectivity in whether the gases being mixed are the same or different, and no subjectivity in what the change of entropy is in either case. The paradox is that it does not feel right that identity makes an objective difference between the two cases, but the empirically-demonstrable distinction between fermions and bosons shows that this intuition does not hold in general. I believe Von Neumann came up with a QM resolution of the paradox.
- jbay808 4y ago
  > are there any cases where it would it be impossible to empirically demonstrate that at least the proponent of the lower figure was wrong? Isn't it more interesting to examine a situation where it would be possible to empirically demonstrate that the proponent of the lower figure was wrong?
  - mannykannot 4y ago
    In that case we could objectively say that one value of S does not yield F for that system (given that F is defined as a maximum), but this would not resolve the general question of subjectivity.