5 ms·

I mostly agree with the first half of what you wrote, especially in terms of pedagogy, but when you delved in the nitty-gritty in the second half, there are a l

I mostly agree with the first half of what you wrote, especially in terms of pedagogy, but when you delved in the nitty-gritty in the second half, there are a lot of mistakes that would impede understanding "advanced" concepts in physics. I believe that is at the root of our disagreement - you are perfectly right about early pedagogy, but understanding (which for me means "capability of advanced applications") requires more. Disdain for the thermodynamic definition limits ones toolbox. (Edit: "disdain" is a bad choice of words) My main claim is: building the notion of entropy without relying on atomic theory is crucial if you want to apply it to "interesting" things like quantum mechanics or black hole physics. Admittedly that does not contradict your earlier posts, but it does significantly contradict the philosophy of your last post. An assortment of issues I have with the second half of what you wrote as related to our discussion of what "understanding entropy" means: - Passwords and cryptographic keys do not have entropy. What we call password entropy is a useful heuristic, but it differs from the rigorous information theory notion of entropy (or the thermodynamic one). I claim that it is important to understand that difference as it leads to understanding the limitations of the heuristic. - The definition I have given is not wrong, it is one of the most standard ways scientist have been defining temperature for the last century. "Perfect insulators do not exist" is an extremely important theorem that you "derive" in theoretical physics and use for a vast array of incredibly important thought experiments on which much of our understanding of nature is build. Not taking seriously the thermodynamic definition of entropy thus limits your power to explain Nature. Edit: "perfect insulators do not exist" is not a statement about how difficult it is to solve some engineering problem. It is an absolute statement with certainty equal to that of "√2 is irrational" or "you can not solve the halting problem". - Temperature is indeed an arbitrary scale, again something that has been established in pretty much any textbook on statistical physics or thermodynamics. At this point I really think it is worthwhile to play the authority card: you are claiming multiple things that contradict standard physics textbooks on which the last hundred years of science are based. - The heat engine efficiency is indeed not something most people care about day to day, but most people do not care about entanglement either. Moreover, I do not care about the heat engine as a question of engineering. However, thought experiments that involve heat engines are crucially important for the derivation of many no-go theorems (including in quantum mechanics). - This is incredibly important: heat is NOT motion. This is incredibly limiting worldview that is insufficient for anything but the simplest of toy problems. Very useful early pedagogical tool, but it has to be discarded early on if you want your student to grow. Sure, it is reasonable to say that it is a bad pedagogy to start with the thermodynamic definition. I teach graduate classes both in physics and in quantum information science (theoretical CS) and I actually follow your way of presenting entropy in the first few lectures. But the atomic theory interpretation of entropy is actually limiting when it comes to advanced physics. Really, the only thing I disagree about with you is your insistence that one does not need the thermodynamic definition in order to "understand" entropy. Please excuse my use of the word "insistence" if that is not the case - the limitation of text-based conversation are creeping in. > I agree, but I think it's much easier to start with the statistical definition and show how the thermodynamic one follows (because that is straightforward) than to start with the thermodynamic definition and try to extend it to non-thermodynamical systems. 100% yes > I've never seen that done successfully. Most of the graduate classes I have taken have done that relatively well, but I would agree with you that undergrad classes need more work in the US. For what is worth, the French Class Prepa curriculum on this topic is absolutely outstanding. I particularly like classes/textbooks that take the differential geometry approach to thermodynamics. As long as the textbook treats stat physics and thermodynamics separately, it is probably a textbook that does it well.

lisper 4y ago
I'm not really relying on atomic theory per se, I'm simply relying on the notion of macroscopic observables that encompass an ensemble of microstates. It just so happens that in thermodynamics those microstates are the positions and velocities of particles, but that that is a reflection of the underlying physics. It's not inherent to the definition. Now, it's possible that my insistence on this is a reflection of my ignorance. I'm not a physicist, I'm a computer scientist. But there are a few things that I know (or think I know) that are at odds with what you are telling me. For example, there is this physical phenomenon called specific heat, which makes temperature scales non-arbitrary. You get to pick two points on the scale, but the remaining points are then determined for you by actual physics. There is a whole field of study called calorimetry which relies on this. Your profile says you are a physicist so you must be aware of these things. I'm not pointing this out to challenge your bona fides, only your pedagogy. You say, for example, "Heat is not motion" and that this is "incredibly important", and that "perfect insulators do not exist is a theorem", but then you don't provide any explanation or references. You may be right about these things, but you haven't actually defended or supported them. You've just put them out there as bald assertions. I don't see how you expect that to be constructive. BTW... > Passwords and cryptographic keys do not have entropy. You are mistaken. Information-theoretic entropy is a generalization of thermodynamic entropy, not a "heuristic". (Imagine how annoying it would be if I stopped there and said nothing further.) Entropy is only a coherent concept relative to a state of (generally incomplete) knowledge. It just so happens that in thermodynamics the incompleteness of our knowledge is determined by physics whereas in cryptography it is determined by withholding information (i.e. keeping secrets). If I flip a coin, and I peek at it but I don't show you, that coin has zero entropy relative to my knowledge, but one bit of entropy relative to yours. The only difference between that and thermodynamics is that in the latter there are O(10^23) coins and it is impossible for anyone to peek at them. My password has zero entropy relative to my knowledge, but (one hopes) a lot of entropy relative to an adversary's knowledge. Indeed, the amount of entropy in my password (measured in bits) is precisely equal to the base-2 logarithm of the number of possible passwords that an adversary would have to try in order to guess it.
- krastanov 4y ago
  > It just so happens that in thermodynamics those microstates are the positions and velocities of particles, but that that is a reflection of the underlying physics. It's not inherent to the definition. Yes, but the rather complicated notion of microstate (or probability or knowledge or information) is not necessary in the thermodynamic definition, which is why it is valuable. > For example, there is this physical phenomenon called specific heat, which makes temperature scales non-arbitrary. If you start by introducing the existence of specific heat (which is a pretty good way to introduce concepts related to temperature both in the statistical and in the thermodynamic case) then you indeed need to choose a canonical temperature scale. Kinda how introducing exponentiation through its derivative provides a canonical choice for the basis of the logarithm. But it is important to appreciate the fact that this canonical choice is somewhat arbitrary: it just makes the equations look more "natural", nothing more. > I'm not pointing this out to challenge your bona fides, only your pedagogy. But I never challenged your assertions on pedagogy. I repeatedly and enthusiastically agreed with them in all my posts. I just strongly disagree with your definition of the word "understand". > You may be right about these things, but you haven't actually defended or supported them. You've just put them out there as bald assertions. I don't see how you expect that to be constructive. Part of it is that I never claimed that understanding entropy is easy. I multiple times referred to the axioms in question by name, but did not claim there is a simple explanation that would let one understand what entropy is. The wiki pages on the topic are quite instructive though: - https://en.wikipedia.org/wiki/Zeroth_law_of_thermodynamics#Foundation_of_temperature https://en.wikipedia.org/wiki/Zeroth_law_of_thermodynamics#F... - https://en.wikipedia.org/wiki/Thermodynamic_temperature https://en.wikipedia.org/wiki/Thermodynamic_temperature - Most importantly, the various statements of the second law. I claim fully understanding entropy requires understanding why these various statements are equivalent: https://en.wikipedia.org/wiki/Second_law_of_thermodynamics#Various_statements_of_the_law https://en.wikipedia.org/wiki/Second_law_of_thermodynamics#V... > Information-theoretic entropy is a generalization of thermodynamic entropy, not a "heuristic". No, they are different concepts, neither one is generalization of the other. That is the point I am making all along. "Password entropy", the way it is commonly used and how I assumed you used it, as in "here is my password, calculate its entropy" is not (information theoretic) entropy, it is just a useful heuristic figure of merit. You can not have entropy without having some probability distribution as you already said in your clarification. And a password is just a single microstate: there is no such thing as entropy of a single microstate, but again, it seems we are in agreement about that. > It just so happens that in thermodynamics the incompleteness of our knowledge is determined by physics whereas. This should say "in statistical physics". Thermodynamic entropy has (superficially) nothing to do with knowledge or information. Few more fun references if you are into extreme mathematical rigor: - https://web.ist.utl.pt/berberan/data/68.pdf https://web.ist.utl.pt/berberan/data/68.pdf - https://en.wikipedia.org/wiki/Constantin_Carath%C3%A9odory#Thermodynamics https://en.wikipedia.org/wiki/Constantin_Carath%C3%A9odory#T... Edit to add: There is a neat "contradiction" between the notions of thermodynamic entropy and statistical entropy. Namely, the notion of ergodicity. Thermodynamic entropy does not let you understand ergodicity or the fact that entropy is permitted to spontaneously drop. Information theoretical approaches do permit you to talk about entropy dropping on its own. While on the topic of pedagogy, discussing this clash is probably quite fruitful. But it is a bit outside my area of expertise. The following is probably a good place from which to start picking terms to google https://math.stackexchange.com/questions/433854/relationship-between-entropy-and-ergodicity https://math.stackexchange.com/questions/433854/relationship...

lisper 4y ago

I'm not really relying on atomic theory per se, I'm simply relying on the notion of macroscopic observables that encompass an ensemble of microstates. It just so happens that in thermodynamics those microstates are the positions and velocities of particles, but that that is a reflection of the underlying physics. It's not inherent to the definition. Now, it's possible that my insistence on this is a reflection of my ignorance. I'm not a physicist, I'm a computer scientist. But there are a few things that I know (or think I know) that are at odds with what you are telling me. For example, there is this physical phenomenon called specific heat, which makes temperature scales non-arbitrary. You get to pick two points on the scale, but the remaining points are then determined for you by actual physics. There is a whole field of study called calorimetry which relies on this. Your profile says you are a physicist so you must be aware of these things. I'm not pointing this out to challenge your bona fides, only your pedagogy. You say, for example, "Heat is not motion" and that this is "incredibly important", and that "perfect insulators do not exist is a theorem", but then you don't provide any explanation or references. You may be right about these things, but you haven't actually defended or supported them. You've just put them out there as bald assertions. I don't see how you expect that to be constructive. BTW... > Passwords and cryptographic keys do not have entropy. You are mistaken. Information-theoretic entropy is a generalization of thermodynamic entropy, not a "heuristic". (Imagine how annoying it would be if I stopped there and said nothing further.) Entropy is only a coherent concept relative to a state of (generally incomplete) knowledge. It just so happens that in thermodynamics the incompleteness of our knowledge is determined by physics whereas in cryptography it is determined by withholding information (i.e. keeping secrets). If I flip a coin, and I peek at it but I don't show you, that coin has zero entropy relative to my knowledge, but one bit of entropy relative to yours. The only difference between that and thermodynamics is that in the latter there are O(10^23) coins and it is impossible for anyone to peek at them. My password has zero entropy relative to my knowledge, but (one hopes) a lot of entropy relative to an adversary's knowledge. Indeed, the amount of entropy in my password (measured in bits) is precisely equal to the base-2 logarithm of the number of possible passwords that an adversary would have to try in order to guess it.

krastanov 4y ago
> It just so happens that in thermodynamics those microstates are the positions and velocities of particles, but that that is a reflection of the underlying physics. It's not inherent to the definition. Yes, but the rather complicated notion of microstate (or probability or knowledge or information) is not necessary in the thermodynamic definition, which is why it is valuable. > For example, there is this physical phenomenon called specific heat, which makes temperature scales non-arbitrary. If you start by introducing the existence of specific heat (which is a pretty good way to introduce concepts related to temperature both in the statistical and in the thermodynamic case) then you indeed need to choose a canonical temperature scale. Kinda how introducing exponentiation through its derivative provides a canonical choice for the basis of the logarithm. But it is important to appreciate the fact that this canonical choice is somewhat arbitrary: it just makes the equations look more "natural", nothing more. > I'm not pointing this out to challenge your bona fides, only your pedagogy. But I never challenged your assertions on pedagogy. I repeatedly and enthusiastically agreed with them in all my posts. I just strongly disagree with your definition of the word "understand". > You may be right about these things, but you haven't actually defended or supported them. You've just put them out there as bald assertions. I don't see how you expect that to be constructive. Part of it is that I never claimed that understanding entropy is easy. I multiple times referred to the axioms in question by name, but did not claim there is a simple explanation that would let one understand what entropy is. The wiki pages on the topic are quite instructive though: - https://en.wikipedia.org/wiki/Zeroth_law_of_thermodynamics#Foundation_of_temperature https://en.wikipedia.org/wiki/Zeroth_law_of_thermodynamics#F... - https://en.wikipedia.org/wiki/Thermodynamic_temperature https://en.wikipedia.org/wiki/Thermodynamic_temperature - Most importantly, the various statements of the second law. I claim fully understanding entropy requires understanding why these various statements are equivalent: https://en.wikipedia.org/wiki/Second_law_of_thermodynamics#Various_statements_of_the_law https://en.wikipedia.org/wiki/Second_law_of_thermodynamics#V... > Information-theoretic entropy is a generalization of thermodynamic entropy, not a "heuristic". No, they are different concepts, neither one is generalization of the other. That is the point I am making all along. "Password entropy", the way it is commonly used and how I assumed you used it, as in "here is my password, calculate its entropy" is not (information theoretic) entropy, it is just a useful heuristic figure of merit. You can not have entropy without having some probability distribution as you already said in your clarification. And a password is just a single microstate: there is no such thing as entropy of a single microstate, but again, it seems we are in agreement about that. > It just so happens that in thermodynamics the incompleteness of our knowledge is determined by physics whereas. This should say "in statistical physics". Thermodynamic entropy has (superficially) nothing to do with knowledge or information. Few more fun references if you are into extreme mathematical rigor: - https://web.ist.utl.pt/berberan/data/68.pdf https://web.ist.utl.pt/berberan/data/68.pdf - https://en.wikipedia.org/wiki/Constantin_Carath%C3%A9odory#Thermodynamics https://en.wikipedia.org/wiki/Constantin_Carath%C3%A9odory#T... Edit to add: There is a neat "contradiction" between the notions of thermodynamic entropy and statistical entropy. Namely, the notion of ergodicity. Thermodynamic entropy does not let you understand ergodicity or the fact that entropy is permitted to spontaneously drop. Information theoretical approaches do permit you to talk about entropy dropping on its own. While on the topic of pedagogy, discussing this clash is probably quite fruitful. But it is a bit outside my area of expertise. The following is probably a good place from which to start picking terms to google https://math.stackexchange.com/questions/433854/relationship-between-entropy-and-ergodicity https://math.stackexchange.com/questions/433854/relationship...