9 ms·

Can someone help me understand what "entropy" is? In physics, my understanding is that the entropy of a collection of dice in a cup is decreasing over time: the

Can someone help me understand what "entropy" is? In physics, my understanding is that the entropy of a collection of dice in a cup is decreasing over time: the entropy of 5 dice in a row is high, and 5 dice in a messy pile is low. In cryptography we say that this has "generated" entropy, implying that the entropy is increasing over time. Are these actually two separate definitions of the same word, or am I misunderstanding something? Is this a "credit"/"debit" type situation? [append] A brief internet search indicates that I'm mistaken about the physical definition of entropy and that it does increase over time, never (universally) decrease. That does clarify the confusion I have here but highlights that I clearly don't have a very good mental picture of what "entropy" is.

sandworm101 6y ago
Entropy of the universe as a whole is increasing. Localized entropy can go up or down as energy is added/removed or reorganized in the system. Water can freeze or thaw. Dice can be random or in a pattern. But eventually all the suns will go out. It is only the overall temperature of the universe, the largest system, that is always increasing. (Sorry, mixed up my increasings and decreasings. How one discusses entropy can change by context.)
amingilani 6y ago
In cryptography, entropy is randomness. The higher degree of randomness something has the more entropy it has. You'll often see entropy expressed in bits. Here's an XKCD comic that uses entropy. It may give you more context: https://xkcd.com/936/ https://xkcd.com/936/
Thorrez 6y ago
The word entropy can refer to 2 separate things, but they are somewhat related: https://en.wikipedia.org/wiki/Entropy_in_thermodynamics_and_information_theory https://en.wikipedia.org/wiki/Entropy_in_thermodynamics_and_... >My greatest concern was what to call it. I thought of calling it 'information,' but the word was overly used, so I decided to call it 'uncertainty.' When I discussed it with John von Neumann, he had a better idea. Von Neumann told me, 'You should call it entropy, for two reasons. In the first place your uncertainty function has been used in statistical mechanics under that name, so it already has a name. In the second place, and more important, no one really knows what entropy really is, so in a debate you will always have the advantage.' https://en.wikiquote.org/wiki/Claude_Elwood_Shannon https://en.wikiquote.org/wiki/Claude_Elwood_Shannon

sandworm101 6y ago

Entropy of the universe as a whole is increasing. Localized entropy can go up or down as energy is added/removed or reorganized in the system. Water can freeze or thaw. Dice can be random or in a pattern. But eventually all the suns will go out. It is only the overall temperature of the universe, the largest system, that is always increasing. (Sorry, mixed up my increasings and decreasings. How one discusses entropy can change by context.)

amingilani 6y ago

In cryptography, entropy is randomness. The higher degree of randomness something has the more entropy it has. You'll often see entropy expressed in bits. Here's an XKCD comic that uses entropy. It may give you more context: https://xkcd.com/936/ https://xkcd.com/936/

Thorrez 6y ago

The word entropy can refer to 2 separate things, but they are somewhat related: https://en.wikipedia.org/wiki/Entropy_in_thermodynamics_and_information_theory https://en.wikipedia.org/wiki/Entropy_in_thermodynamics_and_... >My greatest concern was what to call it. I thought of calling it 'information,' but the word was overly used, so I decided to call it 'uncertainty.' When I discussed it with John von Neumann, he had a better idea. Von Neumann told me, 'You should call it entropy, for two reasons. In the first place your uncertainty function has been used in statistical mechanics under that name, so it already has a name. In the second place, and more important, no one really knows what entropy really is, so in a debate you will always have the advantage.' https://en.wikiquote.org/wiki/Claude_Elwood_Shannon https://en.wikiquote.org/wiki/Claude_Elwood_Shannon