3 ms·

The work to open and close the door can be reduced to an arbitrarily small level, so that doesn't resolve the paradox. The solution is that the demon has to co

The work to open and close the door can be reduced to an arbitrarily small level, so that doesn't resolve the paradox. The solution is that the demon has to collect and then destroy the information relating to its decision whether to open or close the door. It is the cost of destroying this information that balances out the entropy decrease in the system.

Nursie 5y ago
Yeah we've just gone past my level of understanding :) I should probably learn about the information theory aspects here, they seem quite ... derived.
- jfengel 5y ago
  Another way to put it: the decision to open the door requires some minimum unit of energy. That energy is itself at maximum entropy: there's no way to regain it for useful work. The total effect of this is that you can't beat entropy in the long term. That's often expressed in terms of deleting the information, which really does amount to the same thing viewed through a different lens. Looked at that way, the demon would be regaining the (organized) energy it would need to make the next decision. That requires energy input. I find the "deletion" view kinda confusing, but it does work. It's all a matter of where you want to set your zero points.
- crdrost 5y ago
  That's okay, speaking as a physicist the connections between entropy and information are not at all obvious at first glance and you have to “get used to them.” For example, Maxwell's demon makes more sense if you phrase the second law as “there is no physical process which can move heat long-term from a colder temperature to a hotter temperature, without some extra input of energy.” So this says that refrigerators can work, but they only work by consuming power. There is no “spontaneous” flow the other way. Given that statement of the second law, you can very well imagine Maxwell saying, “well if a tiny fluctuation can transfer heat from cold to hot temperature, why can't I just look at these fluctuations and filter out the ones that I don’t want, and then I have a long-term process?” And it is not obvious that every mechanism for filtering is going to require energy input. Feynman gave a better example: a ratchet. A ratchet allows motion one way but not the other. Connect it to a wall between two gases, now the statistical fluctuations where the cold side happens to be pushing more on the wall can move the wall one way, and then the hot side cannot move it back because of the ratchet. You kind of have to peek inside the ratchet and see these springs and bits of metal interlocking, to see how the problem is resolved. The “entropy of erasure” argument is huge because it gives a very different language for all of this, one where these ratchets and filters are viewed as information processors like computers, and run into fundamental bounds on information processing which require the input of energy. So yes, it is several steps of derivation on (the above definition of the 2nd law doesn't even mention entropy!) and is more abstract, which is necessary to be more universal. But if you're not trying to be universal, you can see the ratchet with its asymmetrical cog and the pawl and the spring holding them together, and I just tell you that at nonzero temperature the spring & pawl must be randomly jittery and you can start to realize, “oh, ratchets can only work if they are refrigerated to get that energy out of the spring, once the random jitter is about the size of the cog’s notches, fuggedaboutit!”... Entropy of erasure says basically that this is going to be a universal problem.
  - aeternum 5y ago
    I've always thought of entropy as gradients. The universe somehow started in a non-homogenous state and just due to statistics and random walks, will eventually get to a fully homogeneous state. In the meantime, we can extract useful work from the areas in the universe where there are relatively steep gradients. Energy is what we call these gradients while we can still harvest useful work from them, and entropy is what we call it when near the homogenous state and we can no longer harvest useful work. Is the information erasure perspective compatible with this gradient perspective, or is something wrong about viewing it as gradients?
    - crdrost 5y ago
      Hi! This is a fun question. Certainly, thermal equilibrium involves something called the equipartition theorem, which says that all of the energy is divided equally across all of the degrees of freedom so that each has an average small amount of energy which we call the temperature. (It's hard to define temperature in any other circumstance! But let's not get into ensembles.) The big caution with the gradient idea is that you can get uncomfortably close, if you are not paying attention, to saying that refrigerators are impossible. An energy flow from degree of freedom A to degree of freedom B manages to “slurp up” some of this thermal energy from a system's degrees of freedom and move them to another system, one gradient undoes a flow across a different gradient. And like, this happens all the time in everyone's kitchen. Information erasure looks like this: a bit needs to look like a degree of freedom with some energy stored above the baseline average. It could be energy neutral in that you might have two degrees of freedom, {ifZero, ifOne}, so that a NOT gate doesn't take any energy, but just swaps the energy in the two degrees of freedom. But the energy stored in one or the other needs to be roughly one temperature*, or so. And the reason why is that “equipartition” specifies an average, but there is also a standard deviation: these are the energy fluctuations in thermal equilibrium. So, as a result of Ludwig Boltzmann we know that this is an exponentially-distributed random variable and therefore the standard deviation is equal to the mean. So if you want your bits to not be thermally excited, they need to hold a few temperatures’ worth of energy. Information erasure says that the only circuits on these bits which preserve the energy levels inside them, are reversible circuits like NOT. Irreversible dynamics can look like reversible dynamics plus erasure: in the simplest erasure, we drain both ifZero and ifOne energies into the environment and then we refill ifZero from some energy reservoir. And this says basically that there is nothing much better than that for erasing a bit. So, just like the ratchet had a spring that was kept at low energy in order to facilitate a transfer of energy from cold to hot, and could only work as long as you kept that spring well-refrigerated, Maxwell's demon has a similar “coldness buffer” in terms of these bits that are present in a definite prepared state ahead of time. The information processing step takes it out of that zero state, and if it wants to go back there it has to erase, but that requires dumping some energy out into an environment, and getting energy from some reservoir to compensate. And then Maxwell's demon becomes just a normal refrigerator like the other ones that we know about.

Nursie 5y ago

Yeah we've just gone past my level of understanding :) I should probably learn about the information theory aspects here, they seem quite ... derived.

jfengel 5y ago
Another way to put it: the decision to open the door requires some minimum unit of energy. That energy is itself at maximum entropy: there's no way to regain it for useful work. The total effect of this is that you can't beat entropy in the long term. That's often expressed in terms of deleting the information, which really does amount to the same thing viewed through a different lens. Looked at that way, the demon would be regaining the (organized) energy it would need to make the next decision. That requires energy input. I find the "deletion" view kinda confusing, but it does work. It's all a matter of where you want to set your zero points.
crdrost 5y ago
That's okay, speaking as a physicist the connections between entropy and information are not at all obvious at first glance and you have to “get used to them.” For example, Maxwell's demon makes more sense if you phrase the second law as “there is no physical process which can move heat long-term from a colder temperature to a hotter temperature, without some extra input of energy.” So this says that refrigerators can work, but they only work by consuming power. There is no “spontaneous” flow the other way. Given that statement of the second law, you can very well imagine Maxwell saying, “well if a tiny fluctuation can transfer heat from cold to hot temperature, why can't I just look at these fluctuations and filter out the ones that I don’t want, and then I have a long-term process?” And it is not obvious that every mechanism for filtering is going to require energy input. Feynman gave a better example: a ratchet. A ratchet allows motion one way but not the other. Connect it to a wall between two gases, now the statistical fluctuations where the cold side happens to be pushing more on the wall can move the wall one way, and then the hot side cannot move it back because of the ratchet. You kind of have to peek inside the ratchet and see these springs and bits of metal interlocking, to see how the problem is resolved. The “entropy of erasure” argument is huge because it gives a very different language for all of this, one where these ratchets and filters are viewed as information processors like computers, and run into fundamental bounds on information processing which require the input of energy. So yes, it is several steps of derivation on (the above definition of the 2nd law doesn't even mention entropy!) and is more abstract, which is necessary to be more universal. But if you're not trying to be universal, you can see the ratchet with its asymmetrical cog and the pawl and the spring holding them together, and I just tell you that at nonzero temperature the spring & pawl must be randomly jittery and you can start to realize, “oh, ratchets can only work if they are refrigerated to get that energy out of the spring, once the random jitter is about the size of the cog’s notches, fuggedaboutit!”... Entropy of erasure says basically that this is going to be a universal problem.
- aeternum 5y ago
  I've always thought of entropy as gradients. The universe somehow started in a non-homogenous state and just due to statistics and random walks, will eventually get to a fully homogeneous state. In the meantime, we can extract useful work from the areas in the universe where there are relatively steep gradients. Energy is what we call these gradients while we can still harvest useful work from them, and entropy is what we call it when near the homogenous state and we can no longer harvest useful work. Is the information erasure perspective compatible with this gradient perspective, or is something wrong about viewing it as gradients?
  - crdrost 5y ago
    Hi! This is a fun question. Certainly, thermal equilibrium involves something called the equipartition theorem, which says that all of the energy is divided equally across all of the degrees of freedom so that each has an average small amount of energy which we call the temperature. (It's hard to define temperature in any other circumstance! But let's not get into ensembles.) The big caution with the gradient idea is that you can get uncomfortably close, if you are not paying attention, to saying that refrigerators are impossible. An energy flow from degree of freedom A to degree of freedom B manages to “slurp up” some of this thermal energy from a system's degrees of freedom and move them to another system, one gradient undoes a flow across a different gradient. And like, this happens all the time in everyone's kitchen. Information erasure looks like this: a bit needs to look like a degree of freedom with some energy stored above the baseline average. It could be energy neutral in that you might have two degrees of freedom, {ifZero, ifOne}, so that a NOT gate doesn't take any energy, but just swaps the energy in the two degrees of freedom. But the energy stored in one or the other needs to be roughly one temperature*, or so. And the reason why is that “equipartition” specifies an average, but there is also a standard deviation: these are the energy fluctuations in thermal equilibrium. So, as a result of Ludwig Boltzmann we know that this is an exponentially-distributed random variable and therefore the standard deviation is equal to the mean. So if you want your bits to not be thermally excited, they need to hold a few temperatures’ worth of energy. Information erasure says that the only circuits on these bits which preserve the energy levels inside them, are reversible circuits like NOT. Irreversible dynamics can look like reversible dynamics plus erasure: in the simplest erasure, we drain both ifZero and ifOne energies into the environment and then we refill ifZero from some energy reservoir. And this says basically that there is nothing much better than that for erasing a bit. So, just like the ratchet had a spring that was kept at low energy in order to facilitate a transfer of energy from cold to hot, and could only work as long as you kept that spring well-refrigerated, Maxwell's demon has a similar “coldness buffer” in terms of these bits that are present in a definite prepared state ahead of time. The information processing step takes it out of that zero state, and if it wants to go back there it has to erase, but that requires dumping some energy out into an environment, and getting energy from some reservoir to compensate. And then Maxwell's demon becomes just a normal refrigerator like the other ones that we know about.