3 ms·

Yeah, it is (though the subjectivity might vary). Take a deck of cards, say, a pre-ordered one (technically, you know what microstate it's in) - so let's say t

Yeah, it is (though the subjectivity might vary). Take a deck of cards, say, a pre-ordered one (technically, you know what microstate it's in) - so let's say that its "disorder" is "zero", and your entropy about it is zero. Now shuffle it face down really well, now both its "disorder" and your entropy about it is pretty high (it has a specific microstate that you don't know, and the ensemble of all potential microstates form a macrostate). Now look at this shuffled deck face up again - it's "disorder" is still high, but now your entropy about it is zero again, because you know what microstate it's in.

kergonath 6y ago
Be careful that when you're doing that you basically add variables to your macrostate, to the point where entropy itself is meaningless. This type of example is commonly used in CS, but does not correspond at all to a physical system. Bear in mind that information theory's entropy and statistical Physics' entropy are quite different in the details.
- BlueTemplar 6y ago
  What variables am I adding ? In what details are they different ? http://www.av8n.com/physics/thermo/entropy.html http://www.av8n.com/physics/thermo/entropy.html EDIT : Perhaps more relevant: http://www.av8n.com/physics/thermo/s-relevance.html#ch-s-relevance http://www.av8n.com/physics/thermo/s-relevance.html#ch-s-rel... EDIT2 : A specific example where both the Thermodynamic entropy and the Shannon entropy can be experimentally seen to be equivalent : http://www.av8n.com/physics/thermo/expt-basis.html#sec-demag http://www.av8n.com/physics/thermo/expt-basis.html#sec-demag EDIT3 : Ah, found the chapter discussing this specific point : http://www.av8n.com/physics/thermo/entropy-more.html#sec-s-is-s http://www.av8n.com/physics/thermo/entropy-more.html#sec-s-i...
  - kergonath 6y ago
    > What variables am I adding ? The state of each card. If you define a macrostate in such a way as it can have only one microstate, then yes, entropy is 0. It is also completely artificial. > EDIT : Perhaps more relevant: > http://www.av8n.com/physics/thermo/s-relevance.html#ch-s-rel http://www.av8n.com/physics/thermo/s-relevance.html#ch-s-rel... I am not impressed with that website in general, but in this instance I don't see anything wrong with that section. Note in particular: > Very roughly speaking, the items higher on the list can be assigned to the “information theory” camp, while the items lower on the list can be assigned to the “thermodynamics” camp. However, there is tremendous overlap between the two camps. There is indeed some overlap (I have personally worked for a couple of years on applying information theory to calculate entropy in glass-forming materials), but not enough that you can just apply random concepts from one field to the other. Basically, your interpretation is that "we know a lot about the system, therefore entropy is low", whilst in a physical system it's the other way around: "entropy is low, therefore we know a lot about it". Knowing something does not change the state of the system you are observing. > EDIT2 : A specific example where both the Thermodynamic entropy and the Shannon entropy can be experimentally seen to be equivalent : > http://www.av8n.com/physics/thermo/expt-basis.html#sec-demag http://www.av8n.com/physics/thermo/expt-basis.html#sec-demag Entropy goes to zero when spins align, but it does not mean that entropy is non-zero until we check that the spins are aligned. Entropy itself is whatever it is even before we bother calculating it, and we can estimate it different ways. Sure, we can use Shannon's formula in some cases, which is really just Boltzmann's formula with different units. Entropy is a thermodynamical property of a bit of stuff, regardless of what we know about it. The conservation of Gibbs free energy does not suddenly break down because we stop (or start) looking.
    - BlueTemplar 6y ago
      > The state of each card. If you define a macrostate in such a way as it can have only one microstate, then yes, entropy is 0. It is also completely artificial. Obviously, a deck of cards can be in a lot of different microstates, I'm not sure what do you mean ? Consider that "macrostate" is another name for "distribution" ? > There is indeed some overlap (I have personally worked for a couple of years on applying information theory to calculate entropy in glass-forming materials), but not enough that you can just apply random concepts from one field to the other. Basically, your interpretation is that "we know a lot about the system, therefore entropy is low", whilst in a physical system it's the other way around: "entropy is low, therefore we know a lot about it". Knowing something does not change the state of the system you are observing. This is why having two different terms "entropy" and "lack of information" that correspond to the same underlying physical concept is problematic : it causes this kind of confusion. Knowing a lot about a system is equivalent to having low entropy about it. > Entropy itself is whatever it is even before we bother calculating it, and we can estimate it different ways. [...] Entropy is a thermodynamical property of a bit of stuff, regardless of what we know about it. The conservation of Gibbs free energy does not suddenly break down because we stop (or start) looking. "Looking" (aka "measurement" or "observation") is not a "free" operation. To go back to the example of the deck of cards, the deck would have needed to be prepared beforehand to be "ordered", and the action of "looking" at it and to check in what order each card is in is not "free" either. In the same way, neither constructing an aligned spin system nor checking its alignment are "free". As another example, consider the Szilard engine, which allows us to "extract" energy "from" information : https://www.pnas.org/content/111/38/13786 https://www.pnas.org/content/111/38/13786 (Semi-)Classical entropy is defined in terms of probability, it's a functional that depends on a specific probability distribution (aka macrostate) : it's the expected value (aka weighted average) over the surprisals (aka information contents) of all the possible outcomes (aka microstates). (There's also a generalization of it for the situations where we have entangled microstates and quantum mechanics really come into play.) As far as we know, this model works for all the situations where the two nuclear forces and/or relativistic effects can be neglected.

kergonath 6y ago

Be careful that when you're doing that you basically add variables to your macrostate, to the point where entropy itself is meaningless. This type of example is commonly used in CS, but does not correspond at all to a physical system. Bear in mind that information theory's entropy and statistical Physics' entropy are quite different in the details.

BlueTemplar 6y ago
What variables am I adding ? In what details are they different ? http://www.av8n.com/physics/thermo/entropy.html http://www.av8n.com/physics/thermo/entropy.html EDIT : Perhaps more relevant: http://www.av8n.com/physics/thermo/s-relevance.html#ch-s-relevance http://www.av8n.com/physics/thermo/s-relevance.html#ch-s-rel... EDIT2 : A specific example where both the Thermodynamic entropy and the Shannon entropy can be experimentally seen to be equivalent : http://www.av8n.com/physics/thermo/expt-basis.html#sec-demag http://www.av8n.com/physics/thermo/expt-basis.html#sec-demag EDIT3 : Ah, found the chapter discussing this specific point : http://www.av8n.com/physics/thermo/entropy-more.html#sec-s-is-s http://www.av8n.com/physics/thermo/entropy-more.html#sec-s-i...
- kergonath 6y ago
  > What variables am I adding ? The state of each card. If you define a macrostate in such a way as it can have only one microstate, then yes, entropy is 0. It is also completely artificial. > EDIT : Perhaps more relevant: > http://www.av8n.com/physics/thermo/s-relevance.html#ch-s-rel http://www.av8n.com/physics/thermo/s-relevance.html#ch-s-rel... I am not impressed with that website in general, but in this instance I don't see anything wrong with that section. Note in particular: > Very roughly speaking, the items higher on the list can be assigned to the “information theory” camp, while the items lower on the list can be assigned to the “thermodynamics” camp. However, there is tremendous overlap between the two camps. There is indeed some overlap (I have personally worked for a couple of years on applying information theory to calculate entropy in glass-forming materials), but not enough that you can just apply random concepts from one field to the other. Basically, your interpretation is that "we know a lot about the system, therefore entropy is low", whilst in a physical system it's the other way around: "entropy is low, therefore we know a lot about it". Knowing something does not change the state of the system you are observing. > EDIT2 : A specific example where both the Thermodynamic entropy and the Shannon entropy can be experimentally seen to be equivalent : > http://www.av8n.com/physics/thermo/expt-basis.html#sec-demag http://www.av8n.com/physics/thermo/expt-basis.html#sec-demag Entropy goes to zero when spins align, but it does not mean that entropy is non-zero until we check that the spins are aligned. Entropy itself is whatever it is even before we bother calculating it, and we can estimate it different ways. Sure, we can use Shannon's formula in some cases, which is really just Boltzmann's formula with different units. Entropy is a thermodynamical property of a bit of stuff, regardless of what we know about it. The conservation of Gibbs free energy does not suddenly break down because we stop (or start) looking.
  - BlueTemplar 6y ago
    > The state of each card. If you define a macrostate in such a way as it can have only one microstate, then yes, entropy is 0. It is also completely artificial. Obviously, a deck of cards can be in a lot of different microstates, I'm not sure what do you mean ? Consider that "macrostate" is another name for "distribution" ? > There is indeed some overlap (I have personally worked for a couple of years on applying information theory to calculate entropy in glass-forming materials), but not enough that you can just apply random concepts from one field to the other. Basically, your interpretation is that "we know a lot about the system, therefore entropy is low", whilst in a physical system it's the other way around: "entropy is low, therefore we know a lot about it". Knowing something does not change the state of the system you are observing. This is why having two different terms "entropy" and "lack of information" that correspond to the same underlying physical concept is problematic : it causes this kind of confusion. Knowing a lot about a system is equivalent to having low entropy about it. > Entropy itself is whatever it is even before we bother calculating it, and we can estimate it different ways. [...] Entropy is a thermodynamical property of a bit of stuff, regardless of what we know about it. The conservation of Gibbs free energy does not suddenly break down because we stop (or start) looking. "Looking" (aka "measurement" or "observation") is not a "free" operation. To go back to the example of the deck of cards, the deck would have needed to be prepared beforehand to be "ordered", and the action of "looking" at it and to check in what order each card is in is not "free" either. In the same way, neither constructing an aligned spin system nor checking its alignment are "free". As another example, consider the Szilard engine, which allows us to "extract" energy "from" information : https://www.pnas.org/content/111/38/13786 https://www.pnas.org/content/111/38/13786 (Semi-)Classical entropy is defined in terms of probability, it's a functional that depends on a specific probability distribution (aka macrostate) : it's the expected value (aka weighted average) over the surprisals (aka information contents) of all the possible outcomes (aka microstates). (There's also a generalization of it for the situations where we have entangled microstates and quantum mechanics really come into play.) As far as we know, this model works for all the situations where the two nuclear forces and/or relativistic effects can be neglected.