5 ms·

8 bits, by definition. The crux of information theory is that you can analyze a message without know what it means. Mutual information would help you quantify

8 bits, by definition. The crux of information theory is that you can analyze a message without know what it means. Mutual information would help you quantify how much this changes your mental model of the owner's life. Maybe you already knew about the vacation (you went too?). It would let you quantify how much this change in definitions affected you.

goatlover 7y ago
What makes those bits zeros and ones? What makes the thumb drive an information storage device? Why do we need to know the mental model of the owner's life to infer the information stored? For information to be objective, it needs to be understood as mind-independent feature of the world. If you have to reference human minds to make it meaningful, then it's not objective. It's a cultural construct.
- mattkrause 7y ago
  I took the byte and thumb drive bit literally, but if it were some exotic ternary system, it can hold about ~12.7 bits. If they're decimal digits instead, that sequence has a capacity of 10 bans or 26.6 bits. The name "information theory" seems to throw people; imagine it were called coding theory instead and it might seem clearer that we can talk about the "capacity" of different codes and how different representations interact without worrying too much about what they mean. It's similar to algebra, in that we can make statement about `x + y = y + x` or `n + 0 = n` without worrying about whether x is measured in apples, lightyears, or anything at all.
  - goatlover 7y ago
    Right, but if a future alien archaeologist were to dig up a thumb drive (preserved in a vault) after humans went extinct, what would indicate that it held bits?
    - mattkrause 7y ago
      A 'bit' in information theory is just a unit of measurement, corresponding to the information conveyed by one yes-or-no question (where both answers were previously equally likely). Since it's a fairly natural unit, your alien archaeologist might even use it. However, if they had a weird taboo against binary, they could define a similar concept in other ways. In fact, we humans have too. Bletchley Park used "bans" or hartleys, which is the information carried by a single base-10 digit (without any a priori information). Nats use a base-e representation instead, which ties into statistical physics nicely. There's nothing stopping you from inventing a weird informational unit where 1 corresponds to "the answer to a seven way question were two of the answers are each twice as likely as the rest." So, suppose your alien digs up my thumbdrive. It examines it carefully and notices that each MOSFET is always in one of two states. That suffice to describe its information-carrying capacity in whatever system they use: bits, hartleys, nats, zorblaxen. A similar analysis of their own media would let them say "This thing is so sophisticated, it can hold the work of 10,000 scribes" or "This junk can barely contain a moment of our 16D videos." If your drive were packed with different types of media, they might even discover that some parts of the disk (e.g., ascii text) DON'T use all of the available capacity. None of this, of course, tells them what the data means. For that, you'd need to relate these data to some external source.
    - antepodius 7y ago
      There's two different problems here: how you interpret arbitrary strings of bits and how you interpret arbitrary buts of matter as representing bits or not. Yes, figuring out/assuming that the bit cells in the thumb drive represented bits might be difficult. Then again, if you can look and see what the internal physical structure of this thing is, you might see a pattern of highs and lows. But sure, getting from an inefficient physical structure to a series of digits is a step. But once you're there, maths has things to say about the possible information content of strings of bits.
- ganzuul 7y ago
  In great abstraction, you formulate questions which have a 'yes' or 'no' answer. It might make sense to think of the set of all questions with true or false answers, accompanied by an extremely sparse truth table. This is of course impractical but it does let us think about formal logic.
- hackinthebochs 7y ago
  > If you have to reference human minds to make it meaningful, then it's not objective. It's a cultural construct. The fact that certain proteins, e.g. androgen receptors, alter their shape in the presence of specific androgens, implies that the protein's configuration contains mutual information with the shape of the androgen. But this connection between the androgen and its receptor is independent of minds to recognize the connection.
  - goatlover 7y ago
    Can't this just be explained by chemistry and physics?
    - hackinthebochs 7y ago
      Sure, there's no need to mention information in this example to explain the observed behavior. But it's important to understand information as "merely" another kind of description of a system that may be revelatory depending on the context, rather than a new ontology that is competing to displace our old ontology. So saying the receptor has mutual information with the shape of the androgen is just another description of the system that abstracts over some details and highlights others. Going further with the biology example, the concept of mutual information helps to clarify why some sequence of DNA nucleotides are instructions to create proteins that have a particular shape to interact with this molecule or enzyme or whatever. Without the concept of information, we would need to speak of long causal chains and the evolutionary history of this region of DNA with respect to various environmental influences and so on. The concept of information serves to clarify this connection in a much more direct manner.
      - goatlover 7y ago
        That makes sense, but when you're asking philosophical questions about the ontology of computing and information, then if talk of information is just a shortcut for the messy complicated physical process, one can eliminate it from the fundamental "furniture" of the world. One might respond with a shrug and who cares, but then we have people making rather strong claims about the universe being a computer and bit from it. Similar to mathematical universes instead of just physical stuff.

goatlover 7y ago

What makes those bits zeros and ones? What makes the thumb drive an information storage device? Why do we need to know the mental model of the owner's life to infer the information stored? For information to be objective, it needs to be understood as mind-independent feature of the world. If you have to reference human minds to make it meaningful, then it's not objective. It's a cultural construct.

mattkrause 7y ago
I took the byte and thumb drive bit literally, but if it were some exotic ternary system, it can hold about ~12.7 bits. If they're decimal digits instead, that sequence has a capacity of 10 bans or 26.6 bits. The name "information theory" seems to throw people; imagine it were called coding theory instead and it might seem clearer that we can talk about the "capacity" of different codes and how different representations interact without worrying too much about what they mean. It's similar to algebra, in that we can make statement about `x + y = y + x` or `n + 0 = n` without worrying about whether x is measured in apples, lightyears, or anything at all.
- goatlover 7y ago
  Right, but if a future alien archaeologist were to dig up a thumb drive (preserved in a vault) after humans went extinct, what would indicate that it held bits?
  - mattkrause 7y ago
    A 'bit' in information theory is just a unit of measurement, corresponding to the information conveyed by one yes-or-no question (where both answers were previously equally likely). Since it's a fairly natural unit, your alien archaeologist might even use it. However, if they had a weird taboo against binary, they could define a similar concept in other ways. In fact, we humans have too. Bletchley Park used "bans" or hartleys, which is the information carried by a single base-10 digit (without any a priori information). Nats use a base-e representation instead, which ties into statistical physics nicely. There's nothing stopping you from inventing a weird informational unit where 1 corresponds to "the answer to a seven way question were two of the answers are each twice as likely as the rest." So, suppose your alien digs up my thumbdrive. It examines it carefully and notices that each MOSFET is always in one of two states. That suffice to describe its information-carrying capacity in whatever system they use: bits, hartleys, nats, zorblaxen. A similar analysis of their own media would let them say "This thing is so sophisticated, it can hold the work of 10,000 scribes" or "This junk can barely contain a moment of our 16D videos." If your drive were packed with different types of media, they might even discover that some parts of the disk (e.g., ascii text) DON'T use all of the available capacity. None of this, of course, tells them what the data means. For that, you'd need to relate these data to some external source.
  - antepodius 7y ago
    There's two different problems here: how you interpret arbitrary strings of bits and how you interpret arbitrary buts of matter as representing bits or not. Yes, figuring out/assuming that the bit cells in the thumb drive represented bits might be difficult. Then again, if you can look and see what the internal physical structure of this thing is, you might see a pattern of highs and lows. But sure, getting from an inefficient physical structure to a series of digits is a step. But once you're there, maths has things to say about the possible information content of strings of bits.
ganzuul 7y ago
In great abstraction, you formulate questions which have a 'yes' or 'no' answer. It might make sense to think of the set of all questions with true or false answers, accompanied by an extremely sparse truth table. This is of course impractical but it does let us think about formal logic.
hackinthebochs 7y ago
> If you have to reference human minds to make it meaningful, then it's not objective. It's a cultural construct. The fact that certain proteins, e.g. androgen receptors, alter their shape in the presence of specific androgens, implies that the protein's configuration contains mutual information with the shape of the androgen. But this connection between the androgen and its receptor is independent of minds to recognize the connection.
- goatlover 7y ago
  Can't this just be explained by chemistry and physics?
  - hackinthebochs 7y ago
    Sure, there's no need to mention information in this example to explain the observed behavior. But it's important to understand information as "merely" another kind of description of a system that may be revelatory depending on the context, rather than a new ontology that is competing to displace our old ontology. So saying the receptor has mutual information with the shape of the androgen is just another description of the system that abstracts over some details and highlights others. Going further with the biology example, the concept of mutual information helps to clarify why some sequence of DNA nucleotides are instructions to create proteins that have a particular shape to interact with this molecule or enzyme or whatever. Without the concept of information, we would need to speak of long causal chains and the evolutionary history of this region of DNA with respect to various environmental influences and so on. The concept of information serves to clarify this connection in a much more direct manner.
    - goatlover 7y ago
      That makes sense, but when you're asking philosophical questions about the ontology of computing and information, then if talk of information is just a shortcut for the messy complicated physical process, one can eliminate it from the fundamental "furniture" of the world. One might respond with a shrug and who cares, but then we have people making rather strong claims about the universe being a computer and bit from it. Similar to mathematical universes instead of just physical stuff.