3 ms·
That link has a couple of sentences about “maximum entropy models” and no references. From a quick search it seems that this is a terminology used in some areas
by kgwgk 6y ago
That link has a couple of sentences about “maximum entropy models” and no references. From a quick search it seems that this is a terminology used in some areas thought. It’s not yet completely clear to me what does it mean. (Seems to be about the Ising model which is selected from a large set of ‘similar” Ising models; I would think of “Ising model” as the general thing.)
Statistical mechanical entropy is information entropy (and related to thermodynamical entropy). From the link: “The principle was first expounded by E. T. Jaynes in two papers in 1957[1][2] where he emphasized a natural correspondence between statistical mechanics and information theory. In particular, Jaynes offered a new and very general rationale why the Gibbsian method of statistical mechanics works. He argued that the entropy of statistical mechanics and the information entropy of information theory are basically the same thing. Consequently, statistical mechanics should be seen just as a particular application of a general tool of logical inference and information theory.”
Simplicity is a relative thing, I guess. Let’s say the interestingness/complexity ratio is high.
- andbberger 6y agoStatistical mechanical (Boltzmann) entropy and informational (Shannon) entropy are not the same thing, but they are closely related. Subtleties are important. The Shannon entropy of the distribution of the positions and velocities is precisely the Boltzmann entropy of the same, but there is no statistical mechanical analog to the Shannon entropy of a (non-Boltzmann) distribution, to my knowledge.
- kgwgk 6y ago> Statistical mechanical (Boltzmann) entropy While Boltzmann entropy was first, and an important breakthrough, it has some limitations. Statistical mechanics is actually based on Gibbs/von Neumann entropy, which is equivalent to Shannon entropy. There are many subtleties indeed but I think the relationship between statistical mechanics and information theory may be closer than you think.