4 ms·

I know this is a pretty bold claim, but it is hard to overlook the idea that information theory as proposed by Claude Shannon works as a model of information as

by tmcb 6y ago

I know this is a pretty bold claim, but it is hard to overlook the idea that information theory as proposed by Claude Shannon works as a model of information as related to human cognition, and that it does not imply that information is a fundamental property of the physical universe. There are several interviews in which he mentioned that the understanding of how the human brain works was one of his main inquiries. His work on AI and the Theseus. And the seminal paper on information theory contains a section in which a fidelity evaluation function is defined with relation to the human ear and brain.

nil-sec 6y ago
Shannon wasn’t the first to think about entropy. It’s a general concept that is hugely important in thermodynamics and statistical physics. Information theory itself becomes increasingly more important in fundamental physics, see the firewall problem of black holes. It’s even harder to overlook these ideas and claim the human brain and our ears are somehow relevant to the definition of information and entropy.
- tmcb 6y ago
  Indeed, he was not the first at all. The concept of thermodynamic entropy predates, with a huge margin, its information theoretical counterpart. You make it look like I am saying there is a mystical property of the human brain that backs the validity of information theory, an assertion that has no resemblance at all which what I am attempting to express, and sits in the same category the ones I am trying to debate against do. What I am trying to say is that there is no meaning in a teleological interpretation of information theory. Things like “the purpose of living organisms is to propagate information” contradict information theory because there is no such a concept as absolute information, you always define it on mutual terms.
  - morelisp 6y ago
    > there is no such a concept as absolute information, you always define it on mutual terms. This is true of classical information theory, but not (as I understand it) algorithmic information theory. I think the jury is still out on whether it makes philosophical sense to generalize over universal machines like AIT does, and the practical applications compared to classical information theory seem minimal, from a layman's view it seems like it's been mathematically very fruitful.
    - tmcb 6y ago
      My understanding is that, in order to recognize information, you should at least compare it to the outputs of a source of entropy. It is a common mistake to conflate information and entropy. A bad analogy to mechanics is that the outputs of an entropy source are the frame of reference, the signal is a body and information might be whatever property you wish to analyze, such as velocity or acceleration.
      - nil-sec 6y ago
        So if I’m understanding this correctly you have a relativistic understanding of information in which the zero state is observer dependent? Just for my understanding, consider for example the spin of an electron. It can be in one of two states, up or down. In which scenario am I unclear about the absolute information content of knowing the spin state?
        tmcb 6y ago
        It is really hard for me to reply to that comment. I would say, first, that I need a definition for absolute information, as I have been insisting that information is defined on mutual terms. If we move past that, though, information about the spin state is unclear before measurement.
      - morelisp 6y ago
        > in order to recognize information, you should at least compare it to the outputs of a source of entropy. Again, this is correct for classical information theory which requires some frame of reference for "likelihood". But AIT claims a "global frame" over the minimal representation in all universal machines, the particular choice of machine being at worst constant overhead. You can argue, I think somewhat plausibly, that this frame is still an (inter)subjective frame rather than an objective, absolute one. But if we assume C-T (and we virtually always do), that argument is pretty weak - any other definable frame becomes formally "worse" in that it becomes "merely" a specific case of the universal one.
        6y ago

by tmcb 6y ago

nil-sec 6y ago

Shannon wasn’t the first to think about entropy. It’s a general concept that is hugely important in thermodynamics and statistical physics. Information theory itself becomes increasingly more important in fundamental physics, see the firewall problem of black holes. It’s even harder to overlook these ideas and claim the human brain and our ears are somehow relevant to the definition of information and entropy.

tmcb 6y ago
Indeed, he was not the first at all. The concept of thermodynamic entropy predates, with a huge margin, its information theoretical counterpart. You make it look like I am saying there is a mystical property of the human brain that backs the validity of information theory, an assertion that has no resemblance at all which what I am attempting to express, and sits in the same category the ones I am trying to debate against do. What I am trying to say is that there is no meaning in a teleological interpretation of information theory. Things like “the purpose of living organisms is to propagate information” contradict information theory because there is no such a concept as absolute information, you always define it on mutual terms.
- morelisp 6y ago
  > there is no such a concept as absolute information, you always define it on mutual terms. This is true of classical information theory, but not (as I understand it) algorithmic information theory. I think the jury is still out on whether it makes philosophical sense to generalize over universal machines like AIT does, and the practical applications compared to classical information theory seem minimal, from a layman's view it seems like it's been mathematically very fruitful.
  - tmcb 6y ago
    My understanding is that, in order to recognize information, you should at least compare it to the outputs of a source of entropy. It is a common mistake to conflate information and entropy. A bad analogy to mechanics is that the outputs of an entropy source are the frame of reference, the signal is a body and information might be whatever property you wish to analyze, such as velocity or acceleration.
    - nil-sec 6y ago
      So if I’m understanding this correctly you have a relativistic understanding of information in which the zero state is observer dependent? Just for my understanding, consider for example the spin of an electron. It can be in one of two states, up or down. In which scenario am I unclear about the absolute information content of knowing the spin state?
      - tmcb 6y ago
        It is really hard for me to reply to that comment. I would say, first, that I need a definition for absolute information, as I have been insisting that information is defined on mutual terms. If we move past that, though, information about the spin state is unclear before measurement.
    - morelisp 6y ago
      > in order to recognize information, you should at least compare it to the outputs of a source of entropy. Again, this is correct for classical information theory which requires some frame of reference for "likelihood". But AIT claims a "global frame" over the minimal representation in all universal machines, the particular choice of machine being at worst constant overhead. You can argue, I think somewhat plausibly, that this frame is still an (inter)subjective frame rather than an objective, absolute one. But if we assume C-T (and we virtually always do), that argument is pretty weak - any other definable frame becomes formally "worse" in that it becomes "merely" a specific case of the universal one.
      - 6y ago