6 ms·

> Maybe I have the benefit of giant shoulders, but this seems like a fairly mundane observation. It is not mundane, and it is also not right, at least for entr

> Maybe I have the benefit of giant shoulders, but this seems like a fairly mundane observation. It is not mundane, and it is also not right, at least for entropy in Physics and Thermodynamics. > High-entropy states are those macrostates which have many corresponding microstates. That is how you deduce entropy form a given model. But entropy is also something that we can get from experimental measurements. In this case, the experimental setup does not care about microstates and macrostates, it just has properties like enthalpy, heat capacity and temperature. We can build models after the fact and say that e.g. the entropy of a given gas matches that predicted by our model for ideal gases, or that the entropy of a given solid matches what we know about vibrational entropy. That’s how we say that e.g. hydrogen atoms are indistinguishable. It’s not that they become indistinguishable because we decide so. It’s because we can calculate entropy in both cases and reality does not match the model with distinguishable atoms. > The classification of several microstates into the same macrostate, is this not a distinctly observer-centred function? It seems that way if we consider only our neat models, but it fails to explain why experimental measurements of the entropy of a given materials are consistent and independent of whatever model the people doing the experiment were operating on. Fundamentally, entropy depends on the probability distribution, not the observer.

LudwigNagasena 2y ago
> It is not mundane, and it is also not right, at least for entropy in Physics and Thermodynamics. Articles about MaxEnt thermodynamics by E.T. Jaynes where he talks about the “anthropomorphic” nature of entropy date back to 1960s. How is that not right in physics?
- kergonath 2y ago
  By the look of it, it is another misguided attempt to apply information theory concepts to thermodynamics. Entropy as information is seductive because that way we think we can understand it better, and it looks like it works. But we need to be careful because even though we can get useful insights from it (like Hawking radiation) it’s easy to reach unphysical conclusions. > How is that not right in physics? Why would it be right? Was it used to make predictions that were subsequently verified?
  - kgwgk 2y ago
    That misguided attempt has resulted in full graduate courses and textbooks. Maybe one look is not all that it takes to fully assess its worthiness. https://www.amazon.com/Microphysics-Macrophysics-Applications-Statistical-Mathematical/dp/3540454691 https://www.amazon.com/Microphysics-Macrophysics-Application... https://arxiv.org/pdf/cond-mat/0501322 https://arxiv.org/pdf/cond-mat/0501322
    - kergonath 2y ago
      Plenty of bad stuff made its way into textbooks, I saw some really dodgy stuff at uni. And for every uncontroversial statement we can find a textbook that argues that it is wrong. Sorting the good from the bad is the main point of studying science and it is not easy. What is also important is that approaches that can work in some field or context might be misleading or lead to wrong outcomes in others. Information theory is obviously successful and there is nothing fundamentally wrong with it. Where we should be careful is when we want to apply some reasoning verbatim to a different problem. Sometimes it works, and sometimes it does not. Entropy is a particularly good example. It is abstract enough to be mysterious for a vast majority of the population, hence why these terribly misleading vulgarisation articles pop up so often. Thinking of it in terms of information is sometimes useful, but going from information to knowledge is a leap, and then circling back to Physics is a bit adventurous.
      - kgwgk 2y ago
        Plenty of bad stuff makes its way into hackernews comments as well. Saying that “it could be wrong” doesn’t really support the “it’s wrong” claim, does it?
        kergonath 2y ago
        I did not make that point, though. I rejected an appeal to authority because something was in a textbook. I made no comment about the validity of that person’s work in his field, I just pointed out that this transferability was limited.
markisus 2y ago
Maybe the experimental apparatus is not objective. The quantities we choose to measure are dictated by our psychological and physiological limitations. The volume of a container is not objectively defined. An organism which lives at a faster time scale will see the walls of the container vibrating and oscillating. You must convince the organism to average these measurements over a certain time scale. This averaging throws away information. This is the same with other thermodynamic quantities.
- deleted 2y ago
  [deleted]
- kergonath 2y ago
  > The quantities we choose to measure are dictated by our psychological and physiological limitations. No. The enthalpy changes measured by a calorimeter are not dependent on our psychological limitations. > The volume of a container is not objectively defined. Yes, it is, for any reasonable definition of "objective". We know how to measure lengths, we know how they change when we use different frames of reference so there is no situation in which a volume is subjective. > An organism which lives at a faster time scale will see the walls of the container vibrating and oscillating. This does not matter. We defined a time scale from periodic physical phenomena, and then we know how time changes depending on the frame of reference. There is no subjectivity in this, whatever is doing the measurement has no role in it. Time does not depend on how you feel. It’s Physics, not Psychology. > This is the same with other thermodynamic quantities. No, it’s really not. You seem to know just enough vocabulary to be dangerous and I encourage you to read an introductory Physics textbook.
  - notfed 2y ago
    Sometimes I wish HN had merit badges. Or if you like, a device to measure the amount of information contained within a post.
    - kergonath 2y ago
      I am not sure it would help, I think it would just enhance groupthink. I like how you need to write something obviously stupid or offensive for the downvotes to have a visible effect and that upvotes have no visible effect at all. (Yes, it changes ranking, but there are other factors). People are less prejudiced when they read the comment than if they see that it is already at -3 or +5. Yes, it means that some posts should be more (or less) visible than they are but overall I think it’s a good balance. Besides, I am not that interested in the absolute amount of information in a post. I want information that is relevant to me, and that is very subjective :)
kgwgk 2y ago
> But entropy is also something that we can get from experimental measurements. In this case, the experimental setup does not care about microstates and macrostates, it just has properties like enthalpy, heat capacity and temperature. […] experimental measurements of the entropy of a given materials are consistent and independent of whatever model the people doing the experiment were operating on. Fundamentally, entropy depends on the probability distribution, not the observer. https://bayes.wustl.edu/etj/articles/gibbs.vs.boltzmann.pdf https://bayes.wustl.edu/etj/articles/gibbs.vs.boltzmann.pdf Thermodynamics does have the concept of the entropy of a thermodynamic system; but a given physical system corresponds to many different thermodynamic systems. […] It is clearly meaningless to ask, “What is the entropy of the crystal?” unless we first specify the set of parameters which define its thermodynamic state. […] There is no end to this search for the ultimate "true" entropy until we have reached the point where we control the location of each atom independently. But just at that point the notion of entropy collapses, and we are no longer talking thermodynamics! […] From this we see that entropy is an anthropomorphic concept, not only in the well-known statistical sense that it measures the extent of human ignorance as to the microstate. Even at the purely phenomenological level, entropy is an anthropomorphic concept. For it is a property, not of the physical system, but of the particular experiments you or I choose to perform on it.
- Jensson 2y ago
  Temperature is not just an anthropomorphic concept, and temperature and entropy are directly linked concept you can't have one without the other.
  - deleted 2y ago
    [deleted]
  - mitthrowaway2 2y ago
    Temperature and entropy are directly linked; it follows that temperature is also anthropomorphic. Although I think "observer-dependent" would be a better way to put it; it doesn't have to specifically be relative to a human.
  - BenoitEssiambre 2y ago
deleted 2y ago
[deleted]
UltraSane 2y ago
> That’s how we say that e.g. hydrogen atoms are indistinguishable. It’s not that they become indistinguishable because we decide so. It’s because we can calculate entropy in both cases and reality does not match the model with distinguishable atoms. How does using isotopes that allow atoms to be distinguished affect entropy?
- foobarbecue 2y ago
  Add more particles, get more possible states. Hydrogen is a proton and an electron. Deuterium adds a neutron. More possible states means more entropy, assuming your knowledge of the system stayed constant.
BenoitEssiambre 2y ago
> Fundamentally, entropy depends on the probability distribution, not the observer. Right but a probability distribution represents the uncertainty in an observer so there is no inconsistency here (else you're falling for the Mind Projection Fallacy http://www-biba.inrialpes.fr/Jaynes/cc10k.pdf http://www-biba.inrialpes.fr/Jaynes/cc10k.pdf).
- tsimionescu 2y ago
  This is only true if one assumes that (a) physical processes are fundamentally completely deterministic, and that (b) it is possible to measure the initial state of a system to at least the same level of precision as the factors that influence the outcome. Assumption (a) is currently believed to be false in the case of measuring a quantum system: to the best of our current knowledge, the result of measuring a quantum system is a perfectly random sampling of a probability distribution determined by its wave function. Assumption (b) is also believed to be false, and is certainly known to be false in many practical experiments. Especially given that measurement is a time-consuming process, events that happen at a high frequency may be fundamentally unpredictable on computational grounds (that is, measuring the initial state to enough precision and then computing a probability may be physically impossible in the time that it takes for the event to happen - similar to the concept of computational irreducibility). So, even in theory, the outcomes of certain kinds of experiments are probabilistic in a way that is entirely observer-independent; this is especially true in quantum mechanics, but it is also true in many types of classical experiments.
  - kgwgk 2y ago
    The idea that entropy represents the uncertainty in our description of a system works perfectly well in quantum statistical mechanics (actually better than in classical statistical mechanics where the entropy of a system diverges to minus infinity as the precision of the description increases). The entropy is zero when we have a pure state, a state perfectly defined by a wave function, and greater than zero when we have a mixed state. Regarding your second point, how does the practical impossibility of measuring the initial state invalidate the idea that there is uncertainty about the state?
tbalsam 2y ago
>> The classification of several microstates into the same macrostate, is this not a distinctly observer-centred function? > It seems that way if we consider only our neat models, but it fails to explain why experimental measurements of the entropy of a given materials are consistent and independent of whatever model the people doing the experiment were operating on. Fundamentally, entropy depends on the probability distribution, not the observer. I am not sure that I agree with this -- it feels a little too "neat and tidy" to me. One could argue, for example, that these seemingly-emergent agglomerations of states into these cohesive "macro" units are an emergent property limitations of modelling based of the physical properties of the universe -- but there's no way to necessarily easily tell if this set of behaviors comes from an underlying limitation of _dynamics_ of the underlying state of the system(s) based on the rules or this universe or the limitations of our _capacity to model_ the underlying system based on constraints imposed by the rules of this universe. Entropy by definition involves a relationship (generally at least) between two quantities -- even if implicitly, and oftentimes this is some amount of data and a model used to describe this data. In some senses, being unable to model what we don't know (the unknown unknowns) about this particular kind of emergent state (agglomeration into apparent macrostates) is in some form a necessary and complete requirement for modelling the whole system of possible systems as a whole. As a general rule, I tend to consider all discretizations of things that can be described as apparently-continuous processes inherently "wrong", but still useful. This goes for any kind of definition -- the explicit definitions we use for determining the relationship of entropy between quantities, how we define integers, words we use when relating concepts with seemingly different characteristics (different kinds of uncertainty, for example). We induce a form of loss over the original quantity when doing so -- entropy w.r.t. the underlying model, but this loss is the very thing that also allows us to reason over seemingly previously-unreasonable-about things (for example -- mathematical axioms, etc). These forms of "informational straightjackets" offer tradeoffs in how much we can do with them, vs how much we comprehend them. So, even in this light, the very idea of modelling a thing will always induce some form of loss over what we are working with, meaning that said system can never be used to reason about the properties of itself in a larger form -- never verifiably, ever. Using this induction, we can extend it to attempt to reason then about this meta-level of analysis, showing that because it is indeed a form of model sub-selected from the larger possible space of models, that there is some form of inherent measurable loss, and it cannot be trusted to reason even about itself. And therein lies a contradiction! However, one could postulate that this form of loss results in any model necessarily has some form of "collision" or inherent contradiction in it -- theories like Borsuk-Ulam come to mind, and so we must eventually come to the naked depravity of picking some flawed model to analyze our understanding of the world, and hope to realize along the way that we find a sense of comfort and security in the knowledge that it is built on sand and strings, and its validity may unwind and slip away at any minute. A very curious ideal, indeed.

LudwigNagasena 2y ago

> It is not mundane, and it is also not right, at least for entropy in Physics and Thermodynamics. Articles about MaxEnt thermodynamics by E.T. Jaynes where he talks about the “anthropomorphic” nature of entropy date back to 1960s. How is that not right in physics?

kergonath 2y ago
By the look of it, it is another misguided attempt to apply information theory concepts to thermodynamics. Entropy as information is seductive because that way we think we can understand it better, and it looks like it works. But we need to be careful because even though we can get useful insights from it (like Hawking radiation) it’s easy to reach unphysical conclusions. > How is that not right in physics? Why would it be right? Was it used to make predictions that were subsequently verified?
- kgwgk 2y ago
  That misguided attempt has resulted in full graduate courses and textbooks. Maybe one look is not all that it takes to fully assess its worthiness. https://www.amazon.com/Microphysics-Macrophysics-Applications-Statistical-Mathematical/dp/3540454691 https://www.amazon.com/Microphysics-Macrophysics-Application... https://arxiv.org/pdf/cond-mat/0501322 https://arxiv.org/pdf/cond-mat/0501322
  - kergonath 2y ago
    Plenty of bad stuff made its way into textbooks, I saw some really dodgy stuff at uni. And for every uncontroversial statement we can find a textbook that argues that it is wrong. Sorting the good from the bad is the main point of studying science and it is not easy. What is also important is that approaches that can work in some field or context might be misleading or lead to wrong outcomes in others. Information theory is obviously successful and there is nothing fundamentally wrong with it. Where we should be careful is when we want to apply some reasoning verbatim to a different problem. Sometimes it works, and sometimes it does not. Entropy is a particularly good example. It is abstract enough to be mysterious for a vast majority of the population, hence why these terribly misleading vulgarisation articles pop up so often. Thinking of it in terms of information is sometimes useful, but going from information to knowledge is a leap, and then circling back to Physics is a bit adventurous.
    - kgwgk 2y ago
      Plenty of bad stuff makes its way into hackernews comments as well. Saying that “it could be wrong” doesn’t really support the “it’s wrong” claim, does it?
      - kergonath 2y ago
        I did not make that point, though. I rejected an appeal to authority because something was in a textbook. I made no comment about the validity of that person’s work in his field, I just pointed out that this transferability was limited.

markisus 2y ago

Maybe the experimental apparatus is not objective. The quantities we choose to measure are dictated by our psychological and physiological limitations. The volume of a container is not objectively defined. An organism which lives at a faster time scale will see the walls of the container vibrating and oscillating. You must convince the organism to average these measurements over a certain time scale. This averaging throws away information. This is the same with other thermodynamic quantities.

deleted 2y ago
[deleted]
kergonath 2y ago
> The quantities we choose to measure are dictated by our psychological and physiological limitations. No. The enthalpy changes measured by a calorimeter are not dependent on our psychological limitations. > The volume of a container is not objectively defined. Yes, it is, for any reasonable definition of "objective". We know how to measure lengths, we know how they change when we use different frames of reference so there is no situation in which a volume is subjective. > An organism which lives at a faster time scale will see the walls of the container vibrating and oscillating. This does not matter. We defined a time scale from periodic physical phenomena, and then we know how time changes depending on the frame of reference. There is no subjectivity in this, whatever is doing the measurement has no role in it. Time does not depend on how you feel. It’s Physics, not Psychology. > This is the same with other thermodynamic quantities. No, it’s really not. You seem to know just enough vocabulary to be dangerous and I encourage you to read an introductory Physics textbook.
- notfed 2y ago
  Sometimes I wish HN had merit badges. Or if you like, a device to measure the amount of information contained within a post.
  - kergonath 2y ago
    I am not sure it would help, I think it would just enhance groupthink. I like how you need to write something obviously stupid or offensive for the downvotes to have a visible effect and that upvotes have no visible effect at all. (Yes, it changes ranking, but there are other factors). People are less prejudiced when they read the comment than if they see that it is already at -3 or +5. Yes, it means that some posts should be more (or less) visible than they are but overall I think it’s a good balance. Besides, I am not that interested in the absolute amount of information in a post. I want information that is relevant to me, and that is very subjective :)

kgwgk 2y ago

> But entropy is also something that we can get from experimental measurements. In this case, the experimental setup does not care about microstates and macrostates, it just has properties like enthalpy, heat capacity and temperature. […] experimental measurements of the entropy of a given materials are consistent and independent of whatever model the people doing the experiment were operating on. Fundamentally, entropy depends on the probability distribution, not the observer. https://bayes.wustl.edu/etj/articles/gibbs.vs.boltzmann.pdf https://bayes.wustl.edu/etj/articles/gibbs.vs.boltzmann.pdf Thermodynamics does have the concept of the entropy of a thermodynamic system; but a given physical system corresponds to many different thermodynamic systems. […] It is clearly meaningless to ask, “What is the entropy of the crystal?” unless we first specify the set of parameters which define its thermodynamic state. […] There is no end to this search for the ultimate "true" entropy until we have reached the point where we control the location of each atom independently. But just at that point the notion of entropy collapses, and we are no longer talking thermodynamics! […] From this we see that entropy is an anthropomorphic concept, not only in the well-known statistical sense that it measures the extent of human ignorance as to the microstate. Even at the purely phenomenological level, entropy is an anthropomorphic concept. For it is a property, not of the physical system, but of the particular experiments you or I choose to perform on it.

Jensson 2y ago
Temperature is not just an anthropomorphic concept, and temperature and entropy are directly linked concept you can't have one without the other.
- deleted 2y ago
  [deleted]
- mitthrowaway2 2y ago
  Temperature and entropy are directly linked; it follows that temperature is also anthropomorphic. Although I think "observer-dependent" would be a better way to put it; it doesn't have to specifically be relative to a human.
- BenoitEssiambre 2y ago

deleted 2y ago

[deleted]

UltraSane 2y ago

> That’s how we say that e.g. hydrogen atoms are indistinguishable. It’s not that they become indistinguishable because we decide so. It’s because we can calculate entropy in both cases and reality does not match the model with distinguishable atoms. How does using isotopes that allow atoms to be distinguished affect entropy?

foobarbecue 2y ago
Add more particles, get more possible states. Hydrogen is a proton and an electron. Deuterium adds a neutron. More possible states means more entropy, assuming your knowledge of the system stayed constant.

BenoitEssiambre 2y ago

> Fundamentally, entropy depends on the probability distribution, not the observer. Right but a probability distribution represents the uncertainty in an observer so there is no inconsistency here (else you're falling for the Mind Projection Fallacy http://www-biba.inrialpes.fr/Jaynes/cc10k.pdf http://www-biba.inrialpes.fr/Jaynes/cc10k.pdf).

tsimionescu 2y ago
This is only true if one assumes that (a) physical processes are fundamentally completely deterministic, and that (b) it is possible to measure the initial state of a system to at least the same level of precision as the factors that influence the outcome. Assumption (a) is currently believed to be false in the case of measuring a quantum system: to the best of our current knowledge, the result of measuring a quantum system is a perfectly random sampling of a probability distribution determined by its wave function. Assumption (b) is also believed to be false, and is certainly known to be false in many practical experiments. Especially given that measurement is a time-consuming process, events that happen at a high frequency may be fundamentally unpredictable on computational grounds (that is, measuring the initial state to enough precision and then computing a probability may be physically impossible in the time that it takes for the event to happen - similar to the concept of computational irreducibility). So, even in theory, the outcomes of certain kinds of experiments are probabilistic in a way that is entirely observer-independent; this is especially true in quantum mechanics, but it is also true in many types of classical experiments.
- kgwgk 2y ago
  The idea that entropy represents the uncertainty in our description of a system works perfectly well in quantum statistical mechanics (actually better than in classical statistical mechanics where the entropy of a system diverges to minus infinity as the precision of the description increases). The entropy is zero when we have a pure state, a state perfectly defined by a wave function, and greater than zero when we have a mixed state. Regarding your second point, how does the practical impossibility of measuring the initial state invalidate the idea that there is uncertainty about the state?

tbalsam 2y ago

>> The classification of several microstates into the same macrostate, is this not a distinctly observer-centred function? > It seems that way if we consider only our neat models, but it fails to explain why experimental measurements of the entropy of a given materials are consistent and independent of whatever model the people doing the experiment were operating on. Fundamentally, entropy depends on the probability distribution, not the observer. I am not sure that I agree with this -- it feels a little too "neat and tidy" to me. One could argue, for example, that these seemingly-emergent agglomerations of states into these cohesive "macro" units are an emergent property limitations of modelling based of the physical properties of the universe -- but there's no way to necessarily easily tell if this set of behaviors comes from an underlying limitation of _dynamics_ of the underlying state of the system(s) based on the rules or this universe or the limitations of our _capacity to model_ the underlying system based on constraints imposed by the rules of this universe. Entropy by definition involves a relationship (generally at least) between two quantities -- even if implicitly, and oftentimes this is some amount of data and a model used to describe this data. In some senses, being unable to model what we don't know (the unknown unknowns) about this particular kind of emergent state (agglomeration into apparent macrostates) is in some form a necessary and complete requirement for modelling the whole system of possible systems as a whole. As a general rule, I tend to consider all discretizations of things that can be described as apparently-continuous processes inherently "wrong", but still useful. This goes for any kind of definition -- the explicit definitions we use for determining the relationship of entropy between quantities, how we define integers, words we use when relating concepts with seemingly different characteristics (different kinds of uncertainty, for example). We induce a form of loss over the original quantity when doing so -- entropy w.r.t. the underlying model, but this loss is the very thing that also allows us to reason over seemingly previously-unreasonable-about things (for example -- mathematical axioms, etc). These forms of "informational straightjackets" offer tradeoffs in how much we can do with them, vs how much we comprehend them. So, even in this light, the very idea of modelling a thing will always induce some form of loss over what we are working with, meaning that said system can never be used to reason about the properties of itself in a larger form -- never verifiably, ever. Using this induction, we can extend it to attempt to reason then about this meta-level of analysis, showing that because it is indeed a form of model sub-selected from the larger possible space of models, that there is some form of inherent measurable loss, and it cannot be trusted to reason even about itself. And therein lies a contradiction! However, one could postulate that this form of loss results in any model necessarily has some form of "collision" or inherent contradiction in it -- theories like Borsuk-Ulam come to mind, and so we must eventually come to the naked depravity of picking some flawed model to analyze our understanding of the world, and hope to realize along the way that we find a sense of comfort and security in the knowledge that it is built on sand and strings, and its validity may unwind and slip away at any minute. A very curious ideal, indeed.