3 ms·

This is still just talking about a property of information that resembles one aspect of what's meant by "meaning". This is all still purely structural and inde

This is still just talking about a property of information that resembles one aspect of what's meant by "meaning". This is all still purely structural and independent of semantic content. The example given above about alternate interpretations of the same bit string on a USB drive gets at the problem. Semantics is about the process of interpreting data into something "meaningful" to some particular system (e.g. a human). It's essentially an importing process: converting data to a format that some system knows how to operate on (i.e. "understands). To keep it brief, meaning (or "information" in the non-jargon sense) is about a relation between some data and a system which that data operates on (or the system operates on the data, depending on how you look at it). "information" from information theory is an intrinsic property of data[0]: the information content doesn't change based on what's interpreting it. (At least that's my understanding.) [0] I'm using "data" here to refer to something like a serialization of a system into a form where information-theoretical analysis can be applied to it (e.g. like a bit string).

yters 7y ago
Like I mentioned previously, there are objective sufficient criteria to identify meaning. Even if the interpretive context is subjective. So, we can guarantee true positives, even if we cannot guarantee true negatives.
- westoncb 7y ago
  > there are objective sufficient criteria to identify meaning You have just shifted the problem by now using a very specific, narrow definition of "meaning" rather than doing so with "information". The objective sufficient criteria to identify meaning only work because this definition of meaning refers to an intrinsic property of a system; however, the common usage of "meaning" involves a relation between multiple systems.
  - yters 7y ago
    Not sure what you are saying. Informative content has both intrinsic (randomness deficiency, entropy, kolmogorov sufficient statistic) and relational properties (also randomness deficiency, mutual information, algorithmic mutual information) that can be objectively and quantitatively measured.
    - westoncb 7y ago
      Intrinsic/relational may not be the best way of getting at it after all. It would likely regress into an equally involved question on defining "system". So let me try another approach. We need to distinguish between information and the "semantic content" of information. Here are the definitions I have in mind: - Data: an arbitrary string - Information: a numeric property of some data: INFO(d) - Mutual information: a numeric property of two pieces of data: MI(d1, d2) - Semantic content: the particular "effect on a system"[0] that results when the system internalizes the data: SC(s, d) ('s' denotes the system.) My understanding is that you're claiming MI == SC. There are certain ways we could fiddle with my usage of 'system' above so that it can just be 'data,' too (e.g. representing the system as a string, as with Turing Machines that operate on string reps of Turing Machines). In which case we can at least say that MI and CS both have domains like (data X data), so there is some similarity there. But, even just the fact that MI is going to evaluate to a single numeric quantity, while SC evaluates to an "effect on a system" (i.e. following a state transition), implies that they're referring to different things. I think it's also clear that it would be possible for MI(d1, d2) to evaluate to the same number as MI(d3, d4), even though SC(d1, d2) !== SC(d3, d4), since all the informational relations could be equivalent (between (d1, d2) and (d3, d4)), and yet the particular state transitions followed could differ. [0] We can make this more precise. For instance, in the language of automata theory it would be something like following a particular state transition. Edit: I should clarify one implicit thing here: this hinges on an assumption that my definition of "semantic content" would be a satisfactory match to common usage of the phrase for most people. Happy to hear an alternate that also captures common usage, and I can expand on my choice of def.
      - yters 7y ago
        Great definitions. I would make a distinction between the measure of mutual information and the mutual information itself. The former is obviously not the same as the system effect itself. But the latter may be more related to your SC function.
        westoncb 7y ago
        > But the latter may be more related to your SC function Definitely seems related to me. It's an interesting angle on it I hadn't considered before. One idea is that it's due to something like compatibility of formats: in order for some information to be 'meaningful' in the context of some other system, its format has to be something recognizable to that other system; and the ability to recognize implies some commonality[0]. That commonality implies a certain amount of mutual information. I don't feel certain that the mutual information would be sufficient to account for all the effects of semantic content—but I wouldn't be surprised if it was ;) [0] I wonder if this is something that's been studied: mutual information between patterns and recognizers of patterns... I could see there being some lower bound on mutual information.

yters 7y ago

Like I mentioned previously, there are objective sufficient criteria to identify meaning. Even if the interpretive context is subjective. So, we can guarantee true positives, even if we cannot guarantee true negatives.

westoncb 7y ago
> there are objective sufficient criteria to identify meaning You have just shifted the problem by now using a very specific, narrow definition of "meaning" rather than doing so with "information". The objective sufficient criteria to identify meaning only work because this definition of meaning refers to an intrinsic property of a system; however, the common usage of "meaning" involves a relation between multiple systems.
- yters 7y ago
  Not sure what you are saying. Informative content has both intrinsic (randomness deficiency, entropy, kolmogorov sufficient statistic) and relational properties (also randomness deficiency, mutual information, algorithmic mutual information) that can be objectively and quantitatively measured.
  - westoncb 7y ago
    Intrinsic/relational may not be the best way of getting at it after all. It would likely regress into an equally involved question on defining "system". So let me try another approach. We need to distinguish between information and the "semantic content" of information. Here are the definitions I have in mind: - Data: an arbitrary string - Information: a numeric property of some data: INFO(d) - Mutual information: a numeric property of two pieces of data: MI(d1, d2) - Semantic content: the particular "effect on a system"[0] that results when the system internalizes the data: SC(s, d) ('s' denotes the system.) My understanding is that you're claiming MI == SC. There are certain ways we could fiddle with my usage of 'system' above so that it can just be 'data,' too (e.g. representing the system as a string, as with Turing Machines that operate on string reps of Turing Machines). In which case we can at least say that MI and CS both have domains like (data X data), so there is some similarity there. But, even just the fact that MI is going to evaluate to a single numeric quantity, while SC evaluates to an "effect on a system" (i.e. following a state transition), implies that they're referring to different things. I think it's also clear that it would be possible for MI(d1, d2) to evaluate to the same number as MI(d3, d4), even though SC(d1, d2) !== SC(d3, d4), since all the informational relations could be equivalent (between (d1, d2) and (d3, d4)), and yet the particular state transitions followed could differ. [0] We can make this more precise. For instance, in the language of automata theory it would be something like following a particular state transition. Edit: I should clarify one implicit thing here: this hinges on an assumption that my definition of "semantic content" would be a satisfactory match to common usage of the phrase for most people. Happy to hear an alternate that also captures common usage, and I can expand on my choice of def.
    - yters 7y ago
      Great definitions. I would make a distinction between the measure of mutual information and the mutual information itself. The former is obviously not the same as the system effect itself. But the latter may be more related to your SC function.
      - westoncb 7y ago
        > But the latter may be more related to your SC function Definitely seems related to me. It's an interesting angle on it I hadn't considered before. One idea is that it's due to something like compatibility of formats: in order for some information to be 'meaningful' in the context of some other system, its format has to be something recognizable to that other system; and the ability to recognize implies some commonality[0]. That commonality implies a certain amount of mutual information. I don't feel certain that the mutual information would be sufficient to account for all the effects of semantic content—but I wouldn't be surprised if it was ;) [0] I wonder if this is something that's been studied: mutual information between patterns and recognizers of patterns... I could see there being some lower bound on mutual information.