4 ms·

Entropy in physics is usually the Shannon entropy of the probability distribution over system microstates given known temperature and pressure. If the system is

Entropy in physics is usually the Shannon entropy of the probability distribution over system microstates given known temperature and pressure. If the system is in equilibrium then this is objective.

kergonath 2y ago
Entropy in Physics is usually either the Boltzmann or Gibbs entropy, both of whom were dead before Shannon was born.
- enugu 2y ago
  That's not a problem, as the GP's post is trying to state a mathematical relation not a historical attribution. Often newer concepts shed light on older ones. As Baez's article says, Gibbs entropy is Shannon's entropy of an associated distribution(multiplied by the constant k).
  - kergonath 2y ago
    It is a problem because all three come with a bagage. Almost none of the things discussed in this thread are invalid when discussing actual physical entropy even though the equations are superficially similar. And then there are lots of people being confidently wrong because they assume that it’s just one concept. It really is not.
    - enugu 2y ago
      Don't see how the connection is superficial. Even the classical macroscopic definition of entropy as ΔS=∫TdQ can be derived from the information theory perspective as Baez shows in article(using entropy maximizing distributions and Lagrange multipliers). If you have a more specific critique, it would be good to discuss.
      - im3w1l 2y ago
        In classical physics there is no real objective randomness. Particles have a defined position and momentum and those evolve deterministically. If you somehow learned these then the shannon entropy is zero. If entropy is zero then all kinds of things break down. So now you are forced to consider e.g. temperature an impossibility without quantum-derived randomness, even though temperature does not really seem to be a quantum thing.
        kgwgk 2y ago
        > Particles have a defined position and momentum Which we don’t know precisely. Entropy is about not knowing. > If you somehow learned these then the shannon entropy is zero. Minus infinity. Entropy in classical statistical mechanics is proportional to the logarithm of the volume in phase space. (You need an appropriate extension of Shannon’s entropy to continuous distributions.) > So now you are forced to consider e.g. temperature an impossibility without quantum-derived randomness Or you may study statistical mechanics :-)
        kergonath 2y ago
        > Which we don’t know precisely. Entropy is about not knowing. No, it is not about not knowing. This is an instance of the intuition from Shannon’s entropy does not translate to statistical Physics. It is about the number of possible microstates, which is completely different. In Physics, entropy is a property of a bit of matter, it is not related to the observer or their knowledge. We can measure the enthalpy change of a material sample and work out its entropy without knowing a thing about its structure. > Minus infinity. Entropy in classical statistical mechanics is proportional to the logarithm of the volume in phase space. No, 0. In this case, there is a single state with p=1 and and S = - k Σ p ln(p) = 0. This is the same if you consider the phase space because then it is reduced to a single point (you need a bit of distribution theory to prove it rigorously but it is somewhat intuitive). The probability p of an microstate is always between 0 and 1, therefore p ln(p) is always negative and S is always positive. You get the same using Boltzmann’s approach, in which case Ω = 1 and S = k ln(Ω) is also 0. > (You need an appropriate extension of Shannon’s entropy to continuous distributions.) Gibbs’ entropy. > Or you may study statistical mechanics Indeed.

Entropy in physics is usually the Shannon entropy of the probability distribution over system microstates given known temperature and pressure. If the system is in equilibrium then this is objective.

kergonath 2y ago

Entropy in Physics is usually either the Boltzmann or Gibbs entropy, both of whom were dead before Shannon was born.

enugu 2y ago
That's not a problem, as the GP's post is trying to state a mathematical relation not a historical attribution. Often newer concepts shed light on older ones. As Baez's article says, Gibbs entropy is Shannon's entropy of an associated distribution(multiplied by the constant k).
- kergonath 2y ago
  It is a problem because all three come with a bagage. Almost none of the things discussed in this thread are invalid when discussing actual physical entropy even though the equations are superficially similar. And then there are lots of people being confidently wrong because they assume that it’s just one concept. It really is not.
  - enugu 2y ago
    Don't see how the connection is superficial. Even the classical macroscopic definition of entropy as ΔS=∫TdQ can be derived from the information theory perspective as Baez shows in article(using entropy maximizing distributions and Lagrange multipliers). If you have a more specific critique, it would be good to discuss.
    - im3w1l 2y ago
      In classical physics there is no real objective randomness. Particles have a defined position and momentum and those evolve deterministically. If you somehow learned these then the shannon entropy is zero. If entropy is zero then all kinds of things break down. So now you are forced to consider e.g. temperature an impossibility without quantum-derived randomness, even though temperature does not really seem to be a quantum thing.
      - kgwgk 2y ago
        > Particles have a defined position and momentum Which we don’t know precisely. Entropy is about not knowing. > If you somehow learned these then the shannon entropy is zero. Minus infinity. Entropy in classical statistical mechanics is proportional to the logarithm of the volume in phase space. (You need an appropriate extension of Shannon’s entropy to continuous distributions.) > So now you are forced to consider e.g. temperature an impossibility without quantum-derived randomness Or you may study statistical mechanics :-)
        kergonath 2y ago
        > Which we don’t know precisely. Entropy is about not knowing. No, it is not about not knowing. This is an instance of the intuition from Shannon’s entropy does not translate to statistical Physics. It is about the number of possible microstates, which is completely different. In Physics, entropy is a property of a bit of matter, it is not related to the observer or their knowledge. We can measure the enthalpy change of a material sample and work out its entropy without knowing a thing about its structure. > Minus infinity. Entropy in classical statistical mechanics is proportional to the logarithm of the volume in phase space. No, 0. In this case, there is a single state with p=1 and and S = - k Σ p ln(p) = 0. This is the same if you consider the phase space because then it is reduced to a single point (you need a bit of distribution theory to prove it rigorously but it is somewhat intuitive). The probability p of an microstate is always between 0 and 1, therefore p ln(p) is always negative and S is always positive. You get the same using Boltzmann’s approach, in which case Ω = 1 and S = k ln(Ω) is also 0. > (You need an appropriate extension of Shannon’s entropy to continuous distributions.) Gibbs’ entropy. > Or you may study statistical mechanics Indeed.