4 ms·

I'm not really relying on atomic theory per se, I'm simply relying on the notion of macroscopic observables that encompass an ensemble of microstates. It just

by lisper 4y ago

I'm not really relying on atomic theory per se, I'm simply relying on the notion of macroscopic observables that encompass an ensemble of microstates. It just so happens that in thermodynamics those microstates are the positions and velocities of particles, but that that is a reflection of the underlying physics. It's not inherent to the definition. Now, it's possible that my insistence on this is a reflection of my ignorance. I'm not a physicist, I'm a computer scientist. But there are a few things that I know (or think I know) that are at odds with what you are telling me. For example, there is this physical phenomenon called specific heat, which makes temperature scales non-arbitrary. You get to pick two points on the scale, but the remaining points are then determined for you by actual physics. There is a whole field of study called calorimetry which relies on this. Your profile says you are a physicist so you must be aware of these things. I'm not pointing this out to challenge your bona fides, only your pedagogy. You say, for example, "Heat is not motion" and that this is "incredibly important", and that "perfect insulators do not exist is a theorem", but then you don't provide any explanation or references. You may be right about these things, but you haven't actually defended or supported them. You've just put them out there as bald assertions. I don't see how you expect that to be constructive. BTW... > Passwords and cryptographic keys do not have entropy. You are mistaken. Information-theoretic entropy is a generalization of thermodynamic entropy, not a "heuristic". (Imagine how annoying it would be if I stopped there and said nothing further.) Entropy is only a coherent concept relative to a state of (generally incomplete) knowledge. It just so happens that in thermodynamics the incompleteness of our knowledge is determined by physics whereas in cryptography it is determined by withholding information (i.e. keeping secrets). If I flip a coin, and I peek at it but I don't show you, that coin has zero entropy relative to my knowledge, but one bit of entropy relative to yours. The only difference between that and thermodynamics is that in the latter there are O(10^23) coins and it is impossible for anyone to peek at them. My password has zero entropy relative to my knowledge, but (one hopes) a lot of entropy relative to an adversary's knowledge. Indeed, the amount of entropy in my password (measured in bits) is precisely equal to the base-2 logarithm of the number of possible passwords that an adversary would have to try in order to guess it.

krastanov 4y ago
> It just so happens that in thermodynamics those microstates are the positions and velocities of particles, but that that is a reflection of the underlying physics. It's not inherent to the definition. Yes, but the rather complicated notion of microstate (or probability or knowledge or information) is not necessary in the thermodynamic definition, which is why it is valuable. > For example, there is this physical phenomenon called specific heat, which makes temperature scales non-arbitrary. If you start by introducing the existence of specific heat (which is a pretty good way to introduce concepts related to temperature both in the statistical and in the thermodynamic case) then you indeed need to choose a canonical temperature scale. Kinda how introducing exponentiation through its derivative provides a canonical choice for the basis of the logarithm. But it is important to appreciate the fact that this canonical choice is somewhat arbitrary: it just makes the equations look more "natural", nothing more. > I'm not pointing this out to challenge your bona fides, only your pedagogy. But I never challenged your assertions on pedagogy. I repeatedly and enthusiastically agreed with them in all my posts. I just strongly disagree with your definition of the word "understand". > You may be right about these things, but you haven't actually defended or supported them. You've just put them out there as bald assertions. I don't see how you expect that to be constructive. Part of it is that I never claimed that understanding entropy is easy. I multiple times referred to the axioms in question by name, but did not claim there is a simple explanation that would let one understand what entropy is. The wiki pages on the topic are quite instructive though: - https://en.wikipedia.org/wiki/Zeroth_law_of_thermodynamics#Foundation_of_temperature https://en.wikipedia.org/wiki/Zeroth_law_of_thermodynamics#F... - https://en.wikipedia.org/wiki/Thermodynamic_temperature https://en.wikipedia.org/wiki/Thermodynamic_temperature - Most importantly, the various statements of the second law. I claim fully understanding entropy requires understanding why these various statements are equivalent: https://en.wikipedia.org/wiki/Second_law_of_thermodynamics#Various_statements_of_the_law https://en.wikipedia.org/wiki/Second_law_of_thermodynamics#V... > Information-theoretic entropy is a generalization of thermodynamic entropy, not a "heuristic". No, they are different concepts, neither one is generalization of the other. That is the point I am making all along. "Password entropy", the way it is commonly used and how I assumed you used it, as in "here is my password, calculate its entropy" is not (information theoretic) entropy, it is just a useful heuristic figure of merit. You can not have entropy without having some probability distribution as you already said in your clarification. And a password is just a single microstate: there is no such thing as entropy of a single microstate, but again, it seems we are in agreement about that. > It just so happens that in thermodynamics the incompleteness of our knowledge is determined by physics whereas. This should say "in statistical physics". Thermodynamic entropy has (superficially) nothing to do with knowledge or information. Few more fun references if you are into extreme mathematical rigor: - https://web.ist.utl.pt/berberan/data/68.pdf https://web.ist.utl.pt/berberan/data/68.pdf - https://en.wikipedia.org/wiki/Constantin_Carath%C3%A9odory#Thermodynamics https://en.wikipedia.org/wiki/Constantin_Carath%C3%A9odory#T... Edit to add: There is a neat "contradiction" between the notions of thermodynamic entropy and statistical entropy. Namely, the notion of ergodicity. Thermodynamic entropy does not let you understand ergodicity or the fact that entropy is permitted to spontaneously drop. Information theoretical approaches do permit you to talk about entropy dropping on its own. While on the topic of pedagogy, discussing this clash is probably quite fruitful. But it is a bit outside my area of expertise. The following is probably a good place from which to start picking terms to google https://math.stackexchange.com/questions/433854/relationship-between-entropy-and-ergodicity https://math.stackexchange.com/questions/433854/relationship...

by lisper 4y ago

krastanov 4y ago

> It just so happens that in thermodynamics those microstates are the positions and velocities of particles, but that that is a reflection of the underlying physics. It's not inherent to the definition. Yes, but the rather complicated notion of microstate (or probability or knowledge or information) is not necessary in the thermodynamic definition, which is why it is valuable. > For example, there is this physical phenomenon called specific heat, which makes temperature scales non-arbitrary. If you start by introducing the existence of specific heat (which is a pretty good way to introduce concepts related to temperature both in the statistical and in the thermodynamic case) then you indeed need to choose a canonical temperature scale. Kinda how introducing exponentiation through its derivative provides a canonical choice for the basis of the logarithm. But it is important to appreciate the fact that this canonical choice is somewhat arbitrary: it just makes the equations look more "natural", nothing more. > I'm not pointing this out to challenge your bona fides, only your pedagogy. But I never challenged your assertions on pedagogy. I repeatedly and enthusiastically agreed with them in all my posts. I just strongly disagree with your definition of the word "understand". > You may be right about these things, but you haven't actually defended or supported them. You've just put them out there as bald assertions. I don't see how you expect that to be constructive. Part of it is that I never claimed that understanding entropy is easy. I multiple times referred to the axioms in question by name, but did not claim there is a simple explanation that would let one understand what entropy is. The wiki pages on the topic are quite instructive though: - https://en.wikipedia.org/wiki/Zeroth_law_of_thermodynamics#Foundation_of_temperature https://en.wikipedia.org/wiki/Zeroth_law_of_thermodynamics#F... - https://en.wikipedia.org/wiki/Thermodynamic_temperature https://en.wikipedia.org/wiki/Thermodynamic_temperature - Most importantly, the various statements of the second law. I claim fully understanding entropy requires understanding why these various statements are equivalent: https://en.wikipedia.org/wiki/Second_law_of_thermodynamics#Various_statements_of_the_law https://en.wikipedia.org/wiki/Second_law_of_thermodynamics#V... > Information-theoretic entropy is a generalization of thermodynamic entropy, not a "heuristic". No, they are different concepts, neither one is generalization of the other. That is the point I am making all along. "Password entropy", the way it is commonly used and how I assumed you used it, as in "here is my password, calculate its entropy" is not (information theoretic) entropy, it is just a useful heuristic figure of merit. You can not have entropy without having some probability distribution as you already said in your clarification. And a password is just a single microstate: there is no such thing as entropy of a single microstate, but again, it seems we are in agreement about that. > It just so happens that in thermodynamics the incompleteness of our knowledge is determined by physics whereas. This should say "in statistical physics". Thermodynamic entropy has (superficially) nothing to do with knowledge or information. Few more fun references if you are into extreme mathematical rigor: - https://web.ist.utl.pt/berberan/data/68.pdf https://web.ist.utl.pt/berberan/data/68.pdf - https://en.wikipedia.org/wiki/Constantin_Carath%C3%A9odory#Thermodynamics https://en.wikipedia.org/wiki/Constantin_Carath%C3%A9odory#T... Edit to add: There is a neat "contradiction" between the notions of thermodynamic entropy and statistical entropy. Namely, the notion of ergodicity. Thermodynamic entropy does not let you understand ergodicity or the fact that entropy is permitted to spontaneously drop. Information theoretical approaches do permit you to talk about entropy dropping on its own. While on the topic of pedagogy, discussing this clash is probably quite fruitful. But it is a bit outside my area of expertise. The following is probably a good place from which to start picking terms to google https://math.stackexchange.com/questions/433854/relationship-between-entropy-and-ergodicity https://math.stackexchange.com/questions/433854/relationship...