3 ms·

With respect to the author, the article fails to show that in fact “entropy” is related to “complexity” and how the two are related. Entropy should not be tho

With respect to the author, the article fails to show that in fact “entropy” is related to “complexity” and how the two are related. Entropy should not be thought of as “overhead” or “wasted energy,” which is what I believe the author is getting at. Instead, entropy is the tendency to disorganization. So, the analogy could be the more stuff you have, the more disorganized it becomes; but this is a weak analogy in my opinion. Here is a link to another article that discusses the link between complexity and entropy. The two are indeed related: entropy is necessary for complexity to emerge. Entropy is not, as this author suggests, a result of complexity. https://medium.com/@marktraphagen/entropy-and-complexity-the-surprising-paradox-behind-our-universe-60f11409da9b https://medium.com/@marktraphagen/entropy-and-complexity-the...

bob1029 4y ago
> entropy is necessary for complexity to emerge. Something about this particular line does not sit well with me. How would we define the relationship between entropy, complexity and information?
- jschveibinz 4y ago
  I agree it doesn’t feel right. But complexity, like life, does emerge even though the 2nd law holds. It is a matter of scale. Entropy does not mean everything becomes disordered. And now I defer to the physicists, because as an engineer I am going out of my lane…
- pizza 4y ago
  Minor lightbulb went off in my head: you might be interested in slide 17 here, from a presentation on Shape Dynamics [0] tldr: (Shannon) entropy [1]: expected description length of a state of a system (Shape) complexity [0]: a 'clumped-ness' metric: clump sizes / inter-clump separations information: not sure anyone ever really resolved this in a satisfying way :^) [2] [0] https://philphys.net/wp-content/uploads/2019/Barbour-Saig_Summer_School_2019.pdf https://philphys.net/wp-content/uploads/2019/Barbour-Saig_Su... [1] https://physics.stackexchange.com/questions/606722/how-do-different-definitions-of-entropy-connect-with-each-other?noredirect=1&lq=1 https://physics.stackexchange.com/questions/606722/how-do-di... [2] https://monoskop.org/images/2/2f/Shannon_Claude_E_1956_The_Bandwagon.pdf https://monoskop.org/images/2/2f/Shannon_Claude_E_1956_The_B...
- Comevius 4y ago
  Information thermodynamics is what you are looking for, it's the unification of thermodynamics and information theory. Bear with me because I'm not a physicist, but my understanding is that information needs a medium, in which way it is similar to heat, and you can use the same statistical mechanics to describe it, or fluctuation theorem, which is more precise. My understanding is that this is pretty cool stuff that solves Maxwell's demon and also sort of explains biological evolution, because apparently a system responding to it's environment is computation, performed by changing the system's state as a function of a driving signal, which results in memory about the driving signal that can be interpreted as computing a model, a model that can be predictive of the future. Now apparently how predictive that model is equals to how thermodynamically efficient the system is. Even the smallest molecular machines with memory thus must conduct predictive inference to approach maximal energetic efficiency. https://www.researchgate.net/publication/221703137_Thermodynamics_of_Prediction https://www.researchgate.net/publication/221703137_Thermodyn...
- tingletech 4y ago
  thermodynamic entropy is to heat as Shannon entropy is to information?
  - contravariant 4y ago
    Hmm, not entirely sure if those terms fit exactly. It's easier to point out you can go from one to the other by setting to hamiltonian to the negative logarithm of the probability density (or use the Boltzmann distribution to go the other way).

bob1029 4y ago

> entropy is necessary for complexity to emerge. Something about this particular line does not sit well with me. How would we define the relationship between entropy, complexity and information?

jschveibinz 4y ago
I agree it doesn’t feel right. But complexity, like life, does emerge even though the 2nd law holds. It is a matter of scale. Entropy does not mean everything becomes disordered. And now I defer to the physicists, because as an engineer I am going out of my lane…
pizza 4y ago
Minor lightbulb went off in my head: you might be interested in slide 17 here, from a presentation on Shape Dynamics [0] tldr: (Shannon) entropy [1]: expected description length of a state of a system (Shape) complexity [0]: a 'clumped-ness' metric: clump sizes / inter-clump separations information: not sure anyone ever really resolved this in a satisfying way :^) [2] [0] https://philphys.net/wp-content/uploads/2019/Barbour-Saig_Summer_School_2019.pdf https://philphys.net/wp-content/uploads/2019/Barbour-Saig_Su... [1] https://physics.stackexchange.com/questions/606722/how-do-different-definitions-of-entropy-connect-with-each-other?noredirect=1&lq=1 https://physics.stackexchange.com/questions/606722/how-do-di... [2] https://monoskop.org/images/2/2f/Shannon_Claude_E_1956_The_Bandwagon.pdf https://monoskop.org/images/2/2f/Shannon_Claude_E_1956_The_B...
Comevius 4y ago
Information thermodynamics is what you are looking for, it's the unification of thermodynamics and information theory. Bear with me because I'm not a physicist, but my understanding is that information needs a medium, in which way it is similar to heat, and you can use the same statistical mechanics to describe it, or fluctuation theorem, which is more precise. My understanding is that this is pretty cool stuff that solves Maxwell's demon and also sort of explains biological evolution, because apparently a system responding to it's environment is computation, performed by changing the system's state as a function of a driving signal, which results in memory about the driving signal that can be interpreted as computing a model, a model that can be predictive of the future. Now apparently how predictive that model is equals to how thermodynamically efficient the system is. Even the smallest molecular machines with memory thus must conduct predictive inference to approach maximal energetic efficiency. https://www.researchgate.net/publication/221703137_Thermodynamics_of_Prediction https://www.researchgate.net/publication/221703137_Thermodyn...
tingletech 4y ago
thermodynamic entropy is to heat as Shannon entropy is to information?
- contravariant 4y ago
  Hmm, not entirely sure if those terms fit exactly. It's easier to point out you can go from one to the other by setting to hamiltonian to the negative logarithm of the probability density (or use the Boltzmann distribution to go the other way).