7 ms·

It is in the same sense that entropy is a physical thing. https://en.wikipedia.org/wiki/Information_theory https://en.wikipedia.org/wiki/Information_theory

by jakeva 5y ago

kgwgk 5y ago
Which is not, really. At least not as clearly as other things.
- avmich 5y ago
  It's a bit like saying Maxwell equations are not as clear as Newton's laws.
  - kgwgk 5y ago
    “entropy is an anthropomorphic concept, not only in the well-known statistical sense that it measures the extent of human ignorance as to the microstate. Even at the purely phenomenological level, entropy is an anthropomorphic concept. For it is a property, not of the physical system, but of the particular experiments you or I choose to perform on it.” http://www.lptms.u-psud.fr/membres/trizac/Ens/M2MQPL/Jaynes_entropy.pdf http://www.lptms.u-psud.fr/membres/trizac/Ens/M2MQPL/Jaynes_...
TchoBeer 5y ago
I don't know, something about this feels more like an abstraction than a real thing. If you say "the speed of light is the speed limit for information" that doesn't feel like you're saying something about the way the universe works directly.
- krastanov 5y ago
  Pretty much the entirety of modern physics is this type of abstraction (https://en.wikipedia.org/wiki/The_Unreasonable_Effectiveness_of_Mathematics_in_the_Natural_Sciences https://en.wikipedia.org/wiki/The_Unreasonable_Effectiveness...). You are just facing the discomfort felt when your intuition clashes with the tools usually required to make reliable predictions and explanations about the universe. Consider, how is your current discomfort with this statement any different from the discomfort felt through the centuries when ideas like fields instead of forces, entropy, phase spaces, quantum amplitudes / complex numbers, matrix mechanics, spacetime, curvature, etc were being introduced. They are all abstractions that end up being more reliable than our intuition. Whether this "abstraction" is a "real thing" is a question for the philosophers. For me there is no difference between the two.
- jakeva 5y ago
  Sure it's a bit different (especially if you're concerned with how it makes you feel) than, say, describing tectonic plates or the acceleration of a falling body due to earth's gravity but that doesn't make it not a real thing. Quantum spin is one that gets me, or the uncertainty principle. It makes me very uneasy, but whether or not I'm comfortable is irrelevant. Those are to the best of our knowledge actual features of the universe. Abstractions aren't really in the language of the pure sciences. Analogies, metaphors, etc can all serve to help explain but the speed of the propagation of information in this universe is very much defined as the velocity of light in a vacuum in a completely literal sense. There's no abstraction here. Maybe some confusion about what we mean by 'information' but I'm sure there are better resources if you want an afternoon rabbit hole.
galimaufry 5y ago
I'm not sure this answers the question. The wikipedia page is about the entropy of a probability distribution. But the information speed limit is supposed to apply even if everything is totally deterministic. If I write a single, 100% certain message and put it in a spaceship it still cannot go faster than light, even though there is no information transfer (entropy of my message (a constant random variable) is 0). (I'm not saying you are wrong, I am asking to be corrected)
- aeternum 5y ago
  The spaceship itself represents a huge chunk of information (such as the atomic arrangement of the metal atoms making up the bolts).
  - galimaufry 5y ago
    I agree that there is "information" in the colloquial sense there, or even in the Kolmogorov sense. I don't understand how there is information in the entropy sense, because I do not see a random variable anywhere in this story.
    - aeternum 5y ago
      Entropy isn't necessarily random. I think a better description of entropy might be unimportant states. For the rocket, we might care about the total mass and we might even care about the temperature of those bolts. Those parameters represent information. But there's even more information in the rocket, specifically the momentum*position of each of the atoms within the bolt. That quantity for each atom is measurable/knowable and represents information.
    - mannykannot 5y ago
      I believe we are talking about Shannon's direct analogy between information and entropy here. The low probabliity of the atoms of the spaceship being arranged as they are - a specific design of a spaceship - out of all their possible arrangements, is a state of low entropy and high information content. https://en.wikipedia.org/wiki/Entropy_(information_theory) https://en.wikipedia.org/wiki/Entropy_(information_theory)

It is in the same sense that entropy is a physical thing. https://en.wikipedia.org/wiki/Information_theory https://en.wikipedia.org/wiki/Information_theory

by jakeva 5y ago

It is in the same sense that entropy is a physical thing. https://en.wikipedia.org/wiki/Information_theory https://en.wikipedia.org/wiki/Information_theory

kgwgk 5y ago

Which is not, really. At least not as clearly as other things.

avmich 5y ago
It's a bit like saying Maxwell equations are not as clear as Newton's laws.
- kgwgk 5y ago
  “entropy is an anthropomorphic concept, not only in the well-known statistical sense that it measures the extent of human ignorance as to the microstate. Even at the purely phenomenological level, entropy is an anthropomorphic concept. For it is a property, not of the physical system, but of the particular experiments you or I choose to perform on it.” http://www.lptms.u-psud.fr/membres/trizac/Ens/M2MQPL/Jaynes_entropy.pdf http://www.lptms.u-psud.fr/membres/trizac/Ens/M2MQPL/Jaynes_...

TchoBeer 5y ago

I don't know, something about this feels more like an abstraction than a real thing. If you say "the speed of light is the speed limit for information" that doesn't feel like you're saying something about the way the universe works directly.

krastanov 5y ago
Pretty much the entirety of modern physics is this type of abstraction (https://en.wikipedia.org/wiki/The_Unreasonable_Effectiveness_of_Mathematics_in_the_Natural_Sciences https://en.wikipedia.org/wiki/The_Unreasonable_Effectiveness...). You are just facing the discomfort felt when your intuition clashes with the tools usually required to make reliable predictions and explanations about the universe. Consider, how is your current discomfort with this statement any different from the discomfort felt through the centuries when ideas like fields instead of forces, entropy, phase spaces, quantum amplitudes / complex numbers, matrix mechanics, spacetime, curvature, etc were being introduced. They are all abstractions that end up being more reliable than our intuition. Whether this "abstraction" is a "real thing" is a question for the philosophers. For me there is no difference between the two.
jakeva 5y ago
Sure it's a bit different (especially if you're concerned with how it makes you feel) than, say, describing tectonic plates or the acceleration of a falling body due to earth's gravity but that doesn't make it not a real thing. Quantum spin is one that gets me, or the uncertainty principle. It makes me very uneasy, but whether or not I'm comfortable is irrelevant. Those are to the best of our knowledge actual features of the universe. Abstractions aren't really in the language of the pure sciences. Analogies, metaphors, etc can all serve to help explain but the speed of the propagation of information in this universe is very much defined as the velocity of light in a vacuum in a completely literal sense. There's no abstraction here. Maybe some confusion about what we mean by 'information' but I'm sure there are better resources if you want an afternoon rabbit hole.

galimaufry 5y ago

I'm not sure this answers the question. The wikipedia page is about the entropy of a probability distribution. But the information speed limit is supposed to apply even if everything is totally deterministic. If I write a single, 100% certain message and put it in a spaceship it still cannot go faster than light, even though there is no information transfer (entropy of my message (a constant random variable) is 0). (I'm not saying you are wrong, I am asking to be corrected)

aeternum 5y ago
The spaceship itself represents a huge chunk of information (such as the atomic arrangement of the metal atoms making up the bolts).
- galimaufry 5y ago
  I agree that there is "information" in the colloquial sense there, or even in the Kolmogorov sense. I don't understand how there is information in the entropy sense, because I do not see a random variable anywhere in this story.
  - aeternum 5y ago
    Entropy isn't necessarily random. I think a better description of entropy might be unimportant states. For the rocket, we might care about the total mass and we might even care about the temperature of those bolts. Those parameters represent information. But there's even more information in the rocket, specifically the momentum*position of each of the atoms within the bolt. That quantity for each atom is measurable/knowable and represents information.
  - mannykannot 5y ago
    I believe we are talking about Shannon's direct analogy between information and entropy here. The low probabliity of the atoms of the spaceship being arranged as they are - a specific design of a spaceship - out of all their possible arrangements, is a state of low entropy and high information content. https://en.wikipedia.org/wiki/Entropy_(information_theory) https://en.wikipedia.org/wiki/Entropy_(information_theory)