3 ms·

Think minimum description length. Low entropy states require fewer terms to fully describe than high entropy states. This is an objective property of the system

Think minimum description length. Low entropy states require fewer terms to fully describe than high entropy states. This is an objective property of the system.

amelius 1y ago
In a deterministic system you can just use the time as a way to describe a state, if you started from a known state.
sat_solver 1y ago
You're thinking of information entropy, which is not the same concept as entropy in physics. An ice cube in a warm room can be described using a minimum description length as "ice cube in a warm room" (or a crystal structure inside a fluid space), but if you wait until the heat death of the universe, you just have "a warm room" (a smooth fluid space), which will have an even shorter mdl. Von Neuman should never have repurposed the term entropy from physics. Entropy confuses a lot of people, including me.
- nick__m 1y ago
  And somewhat surprisingly the heat death of the universe is the maximal entropy state. Because there are an infinite number of microstates (all the particles are interchangeable) that lead to the same macrostate: nothing happening for ever!
- hackinthebochs 1y ago
  Maxwell's demon thought experiment implies they are the same concept. Given a complete knowledge of every particle of gas you can in principle create unphysical low entropy distributions of the particles. This[1] goes into more detail. [1] https://en.wikipedia.org/wiki/Entropy_in_thermodynamics_and_information_theory#Information_is_physical https://en.wikipedia.org/wiki/Entropy_in_thermodynamics_and_...
  - bavell 1y ago
    A fun visual explanation: https://youtu.be/8Uilw9t-syQ?si=D9sR2YAm40SPFG3a https://youtu.be/8Uilw9t-syQ?si=D9sR2YAm40SPFG3a
zmgsabst 1y ago
“Number of terms” is a human language construct.
- hackinthebochs 1y ago
  No, it's a representation construct, i.e. how to describe some system in a given basis. The basis can be mathematical. Fourier coefficients for example.
  - zmgsabst 1y ago
    Mathematics is a human language. It being a formal language doesn’t change that. Further, it’s not objective: you’re choosing the basis which causes the complexity, but any particular structure can be made simple in some basis.
    - hackinthebochs 1y ago
      Mathematical notation is a human invention, but the structure that mathematics describes is objective. The choice of basis changes the absolute number of terms, but the relative magnitude of terms for a more or less disordered state is generally fixed outside of degenerate cases.
      - zmgsabst 1y ago
        The structure that most words describe is objective, so you haven’t distinguished math as a language. (Nor is mathematics entirely “objective”, eg, axiom of choice.) And the number of terms in your chosen language with your chosen basis isn’t objective: that’s an intrinsic fact to your frame. The complexity of terms is not fixed — that’s simply wrong mathematically. They’re dependent on our chosen basis. Your definition is circular, in that you’re implicitly defining “non-degenerate” as those which make your claim true. You can’t make the whole class simplified at once, but for any state, there exists a basis in which it is simple.
        1y ago

Think minimum description length. Low entropy states require fewer terms to fully describe than high entropy states. This is an objective property of the system.

amelius 1y ago

In a deterministic system you can just use the time as a way to describe a state, if you started from a known state.

sat_solver 1y ago

You're thinking of information entropy, which is not the same concept as entropy in physics. An ice cube in a warm room can be described using a minimum description length as "ice cube in a warm room" (or a crystal structure inside a fluid space), but if you wait until the heat death of the universe, you just have "a warm room" (a smooth fluid space), which will have an even shorter mdl. Von Neuman should never have repurposed the term entropy from physics. Entropy confuses a lot of people, including me.

nick__m 1y ago
And somewhat surprisingly the heat death of the universe is the maximal entropy state. Because there are an infinite number of microstates (all the particles are interchangeable) that lead to the same macrostate: nothing happening for ever!
hackinthebochs 1y ago
Maxwell's demon thought experiment implies they are the same concept. Given a complete knowledge of every particle of gas you can in principle create unphysical low entropy distributions of the particles. This[1] goes into more detail. [1] https://en.wikipedia.org/wiki/Entropy_in_thermodynamics_and_information_theory#Information_is_physical https://en.wikipedia.org/wiki/Entropy_in_thermodynamics_and_...
- bavell 1y ago
  A fun visual explanation: https://youtu.be/8Uilw9t-syQ?si=D9sR2YAm40SPFG3a https://youtu.be/8Uilw9t-syQ?si=D9sR2YAm40SPFG3a

zmgsabst 1y ago

“Number of terms” is a human language construct.

hackinthebochs 1y ago
No, it's a representation construct, i.e. how to describe some system in a given basis. The basis can be mathematical. Fourier coefficients for example.
- zmgsabst 1y ago
  Mathematics is a human language. It being a formal language doesn’t change that. Further, it’s not objective: you’re choosing the basis which causes the complexity, but any particular structure can be made simple in some basis.
  - hackinthebochs 1y ago
    Mathematical notation is a human invention, but the structure that mathematics describes is objective. The choice of basis changes the absolute number of terms, but the relative magnitude of terms for a more or less disordered state is generally fixed outside of degenerate cases.
    - zmgsabst 1y ago
      The structure that most words describe is objective, so you haven’t distinguished math as a language. (Nor is mathematics entirely “objective”, eg, axiom of choice.) And the number of terms in your chosen language with your chosen basis isn’t objective: that’s an intrinsic fact to your frame. The complexity of terms is not fixed — that’s simply wrong mathematically. They’re dependent on our chosen basis. Your definition is circular, in that you’re implicitly defining “non-degenerate” as those which make your claim true. You can’t make the whole class simplified at once, but for any state, there exists a basis in which it is simple.
      - 1y ago