5 ms·

Recent work in information theory has formalized the concept of semantic information, where Shannon deliberately avoided semantics, and focused purely on the in

by 2-tpg 6y ago

Recent work in information theory has formalized the concept of semantic information, where Shannon deliberately avoided semantics, and focused purely on the information content. They define semantic information as: Information which helps an object stay away from thermodynamic equilibrium (no different from entropic noise). Information then has meaning for a receiver if it helps them stay alive. The very same message may be completely "meaningless" to another receiver. https://arxiv.org/abs/1806.08053 https://arxiv.org/abs/1806.08053 Semantic information, autonomous agency, and nonequilibrium statistical physics > Shannon information theory provides various measures of so-called "syntactic information", which reflect the amount of statistical correlation between systems. In contrast, the concept of "semantic information" refers to those correlations which carry significance or "meaning" for a given system. Semantic information plays an important role in many fields, including biology, cognitive science, and philosophy, and there has been a long-standing interest in formulating a broadly applicable and formal theory of semantic information. In this paper we introduce such a theory. We define semantic information as the syntactic information that a physical system has about its environment which is causally necessary for the system to maintain its own existence. "Causal necessity" is defined in terms of counter-factual interventions which scramble correlations between the system and its environment, while "maintaining existence" is defined in terms of the system's ability to keep itself in a low entropy state. We also use recent results in nonequilibrium statistical physics to analyze semantic information from a thermodynamic point of view. Our framework is grounded in the intrinsic dynamics of a system coupled to an environment, and is applicable to any physical system, living or otherwise. It leads to formal definitions of several concepts that have been intuitively understood to be related to semantic information, including "value of information", "semantic content", and "agency".

yters 6y ago
What about webpages on suicide techniques? Do those count as information?
- mhh__ 6y ago
  I guess death could be viewed as the ultimate move away from the equilibrium (of being alive)?
  - yters 6y ago
    Doesn't death allow the body to achieve maximal entropy? Also, this conversation here has nothing to do with keeping me alive, yet I find it informative. Seems the concept of sematic information is missing something, or a lot.
    - 2-tpg 6y ago
      For me, semantic information is information which allows you to predict future world states, or remove uncertainty about current world states, while optimizing for your utility function. If your utility function is to remove all sense data, then a website with suicide techniques helps you predict future world states (if X then Death), and aligns with your utility function, so it is meaningful information for you. But then you get into heterophenomenology territory: someone may state they are not hungry, while desperate for sustenance (or vice versa). People may temporarily act counter to their typical utility function. Do we want to allow such craziness to influence the formal value of semantic information? > has nothing to do with keeping me alive, yet I find it informative. Imagine having to read the exact same information every day. It won't allow you to adapt to a changing environment or be exposed to new ideas, so you can form new ideas yourself. You'd be stuck in a rut, and as good as non-alive. Soon you'll probably perish out of boredom.
      - yters 6y ago
        I spend long periods of time without ingesting sources of information without getting bored and literally dying. Conversely, if I was constantly bombarded with useful information, I would still go crazy, and that could very well lead to literal death. I think staying alive is one piece of meaningful information out of a very large set of meaningful information, most of which has nothing to do with staying alive. In other words, meaningfulness is orthogonal to staying alive.
totemandtoken 6y ago
You might be interested in this [1]. It sounds like you might be more knowledgeable about this stuff than I am, but Shannon did have some passing interest in his definition of information and 'structural' information. [1] https://en.wikipedia.org/wiki/Macy_conferences https://en.wikipedia.org/wiki/Macy_conferences
hackinthebochs 6y ago
While their definition is in the ballpark, its not general enough to handle cases where survival isn't on the line, or communication between non-living systems. A similar effort is Haig's paper which is broadly compatible with the paper you link but is generic enough to capture semantics broadly: http://philsci-archive.pitt.edu/13287/1/Strange http://philsci-archive.pitt.edu/13287/1/Strange Inversion and Making sense.pdf >An interpreter is a mechanism that uses information in choice. The capabilities of the interpreter couple an entropy of inputs (uncertainty) to an entropy of outputs (indecision). The first entropy is dispelled by observation (input of information ). The second entropy is dispelled by choice of action (output of decision). I propose that an interpreter’s response to inputs (information) be considered the meaning of the information for the interpreter. In this conceptual framework, the designed or evolved mechanisms of interpreters provide the much debated link between Shannon information and semantics.
- eternalban 6y ago
  It's a bit absurdly reductionist, imo. Not all information is acted upon, and not all actions are sense-response, if we're talking 'starting with Biology'. In any event, if that thesis is in agreement with the prelude's "there can be competence without comprehension", then it follows that without the need for comprehension there are no requirement for any sense of "meaning" in the "competent mechanism". Meaning, as we humans experience it, is fully bound up with consciousness and sense of self. And sometimes it takes multiple 'readings' of the same input for us to arrive at "comprehension". If we resort to black box modeling of sentient organisms, which is fine, we should not be projecting our experience of the phenomena of consciousness and sense of self (agency) to black boxes that can already be modeled as computation. There is zero basis for this, beyond our strong tendency as humans for anthropomorphism. The really interesting question, imho, is whether meaning is secondary to form [which is the prevalent modern orthodoxy], or whether meaning can have an independent existence from form. (The fact that multiple forms can embody the same meaning is a data point.) (Agree with your first para.)
  - hackinthebochs 6y ago
    >It's a bit absurdly reductionist, imo. The more reductionist, the better imo. >Meaning, as we humans experience it, is fully bound up with consciousness and sense of self. And it is important to avoid giving meaning an anthropocentric framing which will constrain the analysis too much and thus miss important phenomena. Why should meaning have some qualitative experience associated with it? All sorts of organisms and processes behave in meaningful ways that suggest meaning is embedded in these systems without consciousness. A successful definition of meaning should explain the widespread appearance of meaning in the world. >we should not be projecting our experience of the phenomena of consciousness and sense of self (agency) to black boxes that can already be modeled as computation. But computation is necessarily meaningful! We cannot take some meaningful information, apply an arbitrary sequence of transformations on it, and expect to get meaningful information out. In other words: garbage transformations, garbage out. We see that program can meaningfully transform an input sequence, and that same program will produce garbage with another input sequence. The context of the program, i.e. what input it is operating on, determine what those operations mean, or whether they mean anything at all. The fact that nature picks the useful transformations out of the space of the garbage, or a human programmer picks out the useful transformations needs to be explained. The explanation is that some transformations are meaningful in a given context, and operating in the space of meaning helps guide one through the space of possibility.

by 2-tpg 6y ago

yters 6y ago

What about webpages on suicide techniques? Do those count as information?

mhh__ 6y ago
I guess death could be viewed as the ultimate move away from the equilibrium (of being alive)?
- yters 6y ago
  Doesn't death allow the body to achieve maximal entropy? Also, this conversation here has nothing to do with keeping me alive, yet I find it informative. Seems the concept of sematic information is missing something, or a lot.
  - 2-tpg 6y ago
    For me, semantic information is information which allows you to predict future world states, or remove uncertainty about current world states, while optimizing for your utility function. If your utility function is to remove all sense data, then a website with suicide techniques helps you predict future world states (if X then Death), and aligns with your utility function, so it is meaningful information for you. But then you get into heterophenomenology territory: someone may state they are not hungry, while desperate for sustenance (or vice versa). People may temporarily act counter to their typical utility function. Do we want to allow such craziness to influence the formal value of semantic information? > has nothing to do with keeping me alive, yet I find it informative. Imagine having to read the exact same information every day. It won't allow you to adapt to a changing environment or be exposed to new ideas, so you can form new ideas yourself. You'd be stuck in a rut, and as good as non-alive. Soon you'll probably perish out of boredom.
    - yters 6y ago
      I spend long periods of time without ingesting sources of information without getting bored and literally dying. Conversely, if I was constantly bombarded with useful information, I would still go crazy, and that could very well lead to literal death. I think staying alive is one piece of meaningful information out of a very large set of meaningful information, most of which has nothing to do with staying alive. In other words, meaningfulness is orthogonal to staying alive.

totemandtoken 6y ago

You might be interested in this [1]. It sounds like you might be more knowledgeable about this stuff than I am, but Shannon did have some passing interest in his definition of information and 'structural' information. [1] https://en.wikipedia.org/wiki/Macy_conferences https://en.wikipedia.org/wiki/Macy_conferences

hackinthebochs 6y ago

While their definition is in the ballpark, its not general enough to handle cases where survival isn't on the line, or communication between non-living systems. A similar effort is Haig's paper which is broadly compatible with the paper you link but is generic enough to capture semantics broadly: http://philsci-archive.pitt.edu/13287/1/Strange http://philsci-archive.pitt.edu/13287/1/Strange Inversion and Making sense.pdf >An interpreter is a mechanism that uses information in choice. The capabilities of the interpreter couple an entropy of inputs (uncertainty) to an entropy of outputs (indecision). The first entropy is dispelled by observation (input of information ). The second entropy is dispelled by choice of action (output of decision). I propose that an interpreter’s response to inputs (information) be considered the meaning of the information for the interpreter. In this conceptual framework, the designed or evolved mechanisms of interpreters provide the much debated link between Shannon information and semantics.

eternalban 6y ago
It's a bit absurdly reductionist, imo. Not all information is acted upon, and not all actions are sense-response, if we're talking 'starting with Biology'. In any event, if that thesis is in agreement with the prelude's "there can be competence without comprehension", then it follows that without the need for comprehension there are no requirement for any sense of "meaning" in the "competent mechanism". Meaning, as we humans experience it, is fully bound up with consciousness and sense of self. And sometimes it takes multiple 'readings' of the same input for us to arrive at "comprehension". If we resort to black box modeling of sentient organisms, which is fine, we should not be projecting our experience of the phenomena of consciousness and sense of self (agency) to black boxes that can already be modeled as computation. There is zero basis for this, beyond our strong tendency as humans for anthropomorphism. The really interesting question, imho, is whether meaning is secondary to form [which is the prevalent modern orthodoxy], or whether meaning can have an independent existence from form. (The fact that multiple forms can embody the same meaning is a data point.) (Agree with your first para.)
- hackinthebochs 6y ago
  >It's a bit absurdly reductionist, imo. The more reductionist, the better imo. >Meaning, as we humans experience it, is fully bound up with consciousness and sense of self. And it is important to avoid giving meaning an anthropocentric framing which will constrain the analysis too much and thus miss important phenomena. Why should meaning have some qualitative experience associated with it? All sorts of organisms and processes behave in meaningful ways that suggest meaning is embedded in these systems without consciousness. A successful definition of meaning should explain the widespread appearance of meaning in the world. >we should not be projecting our experience of the phenomena of consciousness and sense of self (agency) to black boxes that can already be modeled as computation. But computation is necessarily meaningful! We cannot take some meaningful information, apply an arbitrary sequence of transformations on it, and expect to get meaningful information out. In other words: garbage transformations, garbage out. We see that program can meaningfully transform an input sequence, and that same program will produce garbage with another input sequence. The context of the program, i.e. what input it is operating on, determine what those operations mean, or whether they mean anything at all. The fact that nature picks the useful transformations out of the space of the garbage, or a human programmer picks out the useful transformations needs to be explained. The explanation is that some transformations are meaningful in a given context, and operating in the space of meaning helps guide one through the space of possibility.