4 ms·

And this is what I prefer too, although with the clarification that its the number of ways that a system can be arranged without changing its macroscopic proper

And this is what I prefer too, although with the clarification that its the number of ways that a system can be arranged without changing its macroscopic properties. Its, unfortunately, not very compatible with Shannon's usage in any but the shallowest sense, which is why it stays firmly in the land of physics.

enugu 2y ago
Assuming each of the N microstates for a given macrostate are equally possible with probability p=1/N, the Shannon Entropy is -Σp.log(p) = -N.p.log(p)=-1.log(1/N)=log(N), which is the physics interpretation. In the continuous version, you would get log(V) where V is the volume in phase space occupied by the microstates for a given macrostate. Liouville's theorem that the volume is conserved in phase space implies that any macroscopic process can only move all the microstates from a macrostate A into a macrostate B only if the volume of B is bigger than the volume of A. This implies that the entropy of B should be bigger than the entropy of A which is the Second Law.
- cubefox 2y ago
  The second law of thermodynamics is time-asymmetric, but the fundamental physical laws are time-symmetric, so from them you can only predict that the entropy of B should be bigger than the entropy of A irrespective of whether B is in the future or the past of A. You need the additional assumption (Past Hypothesis) that the universe started in a low entropy state in order to get the second law of thermodynamics. > If our goal is to predict the future, it suffices to choose a distribution that is uniform in the Liouville measure given to us by classical mechanics (or its quantum analogue). If we want to reconstruct the past, in contrast, we need to conditionalize over trajectories that also started in a low-entropy past state — that the “Past Hypothesis” that is required to get stat mech off the ground in a world governed by time-symmetric fundamental laws. https://www.preposterousuniverse.com/blog/2013/07/09/cosmology-and-the-past-hypothesis/ https://www.preposterousuniverse.com/blog/2013/07/09/cosmolo...
  - kgwgk 2y ago
    The second law of thermodynamics is about systems that are well described by a small set of macroscopic variables. The evolution of an initial macrostate prepared by an experimenter who can control only the macrovariables is reproducible. When a thermodynamical system is prepared in such a reproducible way the preparation is happening in the past, by definition. The second law is about how part of the information that we had about a system - constrained to be in a macrostate - is “lost” when we “forget” the previous state and describe it using just the current macrostate. We know more precisely the past than the future - the previous state is in the past by definition.
kgwgk 2y ago
> not very compatible with Shannon's usage in any but the shallowest sense The connection is not so shallow, there are entire books based on it. “The concept of information, intimately connected with that of probability, gives indeed insight on questions of statistical mechanics such as the meaning of irreversibility. This concept was introduced in statistical physics by Brillouin (1956) and Jaynes (1957) soon after its discovery by Shannon in 1948 (Shannon and Weaver, 1949). An immense literature has since then been published, ranging from research articles to textbooks. The variety of topics that belong to this field of science makes it impossible to give here a bibliography, and special searches are necessary for deepening the understanding of one or another aspect. For tutorial introductions, somewhat more detailed than the present one, see R. Balian (1991-92; 2004).” https://arxiv.org/pdf/cond-mat/0501322 https://arxiv.org/pdf/cond-mat/0501322
- petsfed 2y ago
  I don't dispute that the math is compatible. The problem is the interpretation thereof. When I say "shallowest", I mean the implications of each are very different. Insofar as I'm aware, there is no information-theoretic equivalent to the 2nd or 3rd laws of thermodynamics, so the intuition a student works up from physics about how and why entropy matters just doesn't transfer. Likewise, even if an information science student is well versed in the concept of configuration entropy, that's 15 minutes of one lecture in statistical thermodynamics. There's still the rest of the course to consider.

enugu 2y ago

Assuming each of the N microstates for a given macrostate are equally possible with probability p=1/N, the Shannon Entropy is -Σp.log(p) = -N.p.log(p)=-1.log(1/N)=log(N), which is the physics interpretation. In the continuous version, you would get log(V) where V is the volume in phase space occupied by the microstates for a given macrostate. Liouville's theorem that the volume is conserved in phase space implies that any macroscopic process can only move all the microstates from a macrostate A into a macrostate B only if the volume of B is bigger than the volume of A. This implies that the entropy of B should be bigger than the entropy of A which is the Second Law.

cubefox 2y ago
The second law of thermodynamics is time-asymmetric, but the fundamental physical laws are time-symmetric, so from them you can only predict that the entropy of B should be bigger than the entropy of A irrespective of whether B is in the future or the past of A. You need the additional assumption (Past Hypothesis) that the universe started in a low entropy state in order to get the second law of thermodynamics. > If our goal is to predict the future, it suffices to choose a distribution that is uniform in the Liouville measure given to us by classical mechanics (or its quantum analogue). If we want to reconstruct the past, in contrast, we need to conditionalize over trajectories that also started in a low-entropy past state — that the “Past Hypothesis” that is required to get stat mech off the ground in a world governed by time-symmetric fundamental laws. https://www.preposterousuniverse.com/blog/2013/07/09/cosmology-and-the-past-hypothesis/ https://www.preposterousuniverse.com/blog/2013/07/09/cosmolo...
- kgwgk 2y ago
  The second law of thermodynamics is about systems that are well described by a small set of macroscopic variables. The evolution of an initial macrostate prepared by an experimenter who can control only the macrovariables is reproducible. When a thermodynamical system is prepared in such a reproducible way the preparation is happening in the past, by definition. The second law is about how part of the information that we had about a system - constrained to be in a macrostate - is “lost” when we “forget” the previous state and describe it using just the current macrostate. We know more precisely the past than the future - the previous state is in the past by definition.

kgwgk 2y ago

> not very compatible with Shannon's usage in any but the shallowest sense The connection is not so shallow, there are entire books based on it. “The concept of information, intimately connected with that of probability, gives indeed insight on questions of statistical mechanics such as the meaning of irreversibility. This concept was introduced in statistical physics by Brillouin (1956) and Jaynes (1957) soon after its discovery by Shannon in 1948 (Shannon and Weaver, 1949). An immense literature has since then been published, ranging from research articles to textbooks. The variety of topics that belong to this field of science makes it impossible to give here a bibliography, and special searches are necessary for deepening the understanding of one or another aspect. For tutorial introductions, somewhat more detailed than the present one, see R. Balian (1991-92; 2004).” https://arxiv.org/pdf/cond-mat/0501322 https://arxiv.org/pdf/cond-mat/0501322

petsfed 2y ago
I don't dispute that the math is compatible. The problem is the interpretation thereof. When I say "shallowest", I mean the implications of each are very different. Insofar as I'm aware, there is no information-theoretic equivalent to the 2nd or 3rd laws of thermodynamics, so the intuition a student works up from physics about how and why entropy matters just doesn't transfer. Likewise, even if an information science student is well versed in the concept of configuration entropy, that's 15 minutes of one lecture in statistical thermodynamics. There's still the rest of the course to consider.