3 ms·

I'm a scientist. Information theory always felt curious to me because it adopted terminology and concepts similar to that from statistical mechanics but in prac

I'm a scientist. Information theory always felt curious to me because it adopted terminology and concepts similar to that from statistical mechanics but in practice was always much more difficult due to what could be considered what was "random" vs. what was non-random. After sitting back and thinking about it, what is considered "random" of course is a statement of the probability distribution for the set under consideration. For info theory, that set is some set of strings (say passwords) which is really culturally and historically contigent, while in physics land, the set is microstates that determine macrostates, for which the degeneracy of a macrostate depends on the hamiltonian, full stop. I think mathematically, of course the statements you make are similar (hence why you apply the same prob theory to both) but the systems I study are comparatively easier, while really, the underlying probability distribution for strings is really hard to know in practice because it essentially depends on human history and culture up to that point. For example, in a universe without English, English words (say one-to-oned to a discrete set, so strings of positive integers less than 26 + 10 (including decimal numbers)) would be random. In fact, a universe without that particular Chinese IME, if it was done somewhat differently, then ji32k7au4a83 could be random. It's just interesting to me, another reminder that physics is just that much more easier than anything else.

andrewflnr 8y ago
Hold up. In my physics classes, no one ever gave me a straight answer on what constituted a "macrostate". It always sounded arbitrary for similar reasons to the ones you describe for language. Are you telling me it's literally defined by the energy of the system (the Hamiltonian, right?) alone?
- maksimum 8y ago
  Not sure about physics terms, but in combinatorics I believe that an example of macrostate (also called an 'event') would be "2 coins out of 3 landed heads" and the possible microstates (also called 'outcomes') would be THH, HTH, or HHT.
- jhrmnn 8y ago
  A macrostate is any particular probability distribution of microstates. Usually these are picked to reproduce some macroscopic observable, such as temperature, pressure, volume, magnetisation, etc.
- Symmetry 8y ago
  The degeneracy of a macrostate is just a matter of the Hamiltonian but what counts as a macrostate is, in a sense, arbitrary. It's really just a state you, the observer, can distinguish. This would typically involve things like pressure, volume, and temperature but if you developed a new way of measuring the properties of a system suddenly the possible macrostates multiply in number, each contains fewer microstates, and the entropy of the state decreases. Take this far enough and you could create a Maxwell's Demon to extract energy from thermal motion. But while it's subjective in some sense it was later shown that our subjective knowledge of the world is limited by the laws of physics in other ways and perfect subjective knowledge is impossible. So you could say that entropy is a measure of your ignorance about the exact state of the world, which corresponds nicely to the information theory definition. It's just that in physics everyone is in practice going to be using the same pressure, temperature, and volume measurements while in information theory what constitutes a macrostate is very fuzzy.
ordu 8y ago
> It's just interesting to me, another reminder that physics is just that much more easier than anything else. Such a statemenrs make me nervous. In the late XIX century physics professors told their students that they should stop learning physics and go to some other science, because physics is almost complete. It explains almost everything there are some small issues with electromagnetism which will be solved in a few decades and there would be no more work for physicists. No more physics as a science, just engineering. At that time physics seemed much more easier than anything else. Like it seems now for you, I suppose. Though maybe there would be no more Einsteins, and physics really explained almost everything for this time.
- kaybe 8y ago
  No, the point is that in physics you isolate systems and get rid of all the messy stuff. Compare particle physics to cell biology, for example. Of course the first has complicated math and a lot of ressources thrown at it, but in essence these isolate the thing to measure. How would you even do that with cells? A vacuum or laser lab may be complicated or finicky and capricious to work with, but it is nothing compared to a bio lab and the influences on the organisms you try to study. At least bosons don't react differently when you look at them funny or with the time of day (we assume). (addendum: Like lab mice that change massively depending on the handlers.)
mjburgess 8y ago
I take "random" just being a description of the predictor, that they lack sufficient information to make a determinate prediction. ie., I take probability to be only an epistemic description of confidence given information. And therefore randomness just a lack. A physical system may be "ontologically random" in the sense that there is no info its possible to obtain to make a determinate prediction -- but that isn't randomness (which is epistemic). That's "physical informationlessness" which is an (alleged) feature of a physical system that leads to n "inevitable randomness" in our predictions of it.

andrewflnr 8y ago

Hold up. In my physics classes, no one ever gave me a straight answer on what constituted a "macrostate". It always sounded arbitrary for similar reasons to the ones you describe for language. Are you telling me it's literally defined by the energy of the system (the Hamiltonian, right?) alone?

maksimum 8y ago
Not sure about physics terms, but in combinatorics I believe that an example of macrostate (also called an 'event') would be "2 coins out of 3 landed heads" and the possible microstates (also called 'outcomes') would be THH, HTH, or HHT.
jhrmnn 8y ago
A macrostate is any particular probability distribution of microstates. Usually these are picked to reproduce some macroscopic observable, such as temperature, pressure, volume, magnetisation, etc.
Symmetry 8y ago
The degeneracy of a macrostate is just a matter of the Hamiltonian but what counts as a macrostate is, in a sense, arbitrary. It's really just a state you, the observer, can distinguish. This would typically involve things like pressure, volume, and temperature but if you developed a new way of measuring the properties of a system suddenly the possible macrostates multiply in number, each contains fewer microstates, and the entropy of the state decreases. Take this far enough and you could create a Maxwell's Demon to extract energy from thermal motion. But while it's subjective in some sense it was later shown that our subjective knowledge of the world is limited by the laws of physics in other ways and perfect subjective knowledge is impossible. So you could say that entropy is a measure of your ignorance about the exact state of the world, which corresponds nicely to the information theory definition. It's just that in physics everyone is in practice going to be using the same pressure, temperature, and volume measurements while in information theory what constitutes a macrostate is very fuzzy.

ordu 8y ago

> It's just interesting to me, another reminder that physics is just that much more easier than anything else. Such a statemenrs make me nervous. In the late XIX century physics professors told their students that they should stop learning physics and go to some other science, because physics is almost complete. It explains almost everything there are some small issues with electromagnetism which will be solved in a few decades and there would be no more work for physicists. No more physics as a science, just engineering. At that time physics seemed much more easier than anything else. Like it seems now for you, I suppose. Though maybe there would be no more Einsteins, and physics really explained almost everything for this time.

kaybe 8y ago
No, the point is that in physics you isolate systems and get rid of all the messy stuff. Compare particle physics to cell biology, for example. Of course the first has complicated math and a lot of ressources thrown at it, but in essence these isolate the thing to measure. How would you even do that with cells? A vacuum or laser lab may be complicated or finicky and capricious to work with, but it is nothing compared to a bio lab and the influences on the organisms you try to study. At least bosons don't react differently when you look at them funny or with the time of day (we assume). (addendum: Like lab mice that change massively depending on the handlers.)

mjburgess 8y ago

I take "random" just being a description of the predictor, that they lack sufficient information to make a determinate prediction. ie., I take probability to be only an epistemic description of confidence given information. And therefore randomness just a lack. A physical system may be "ontologically random" in the sense that there is no info its possible to obtain to make a determinate prediction -- but that isn't randomness (which is epistemic). That's "physical informationlessness" which is an (alleged) feature of a physical system that leads to n "inevitable randomness" in our predictions of it.