5 ms·

I took the byte and thumb drive bit literally, but if it were some exotic ternary system, it can hold about ~12.7 bits. If they're decimal digits instead, that

I took the byte and thumb drive bit literally, but if it were some exotic ternary system, it can hold about ~12.7 bits. If they're decimal digits instead, that sequence has a capacity of 10 bans or 26.6 bits. The name "information theory" seems to throw people; imagine it were called coding theory instead and it might seem clearer that we can talk about the "capacity" of different codes and how different representations interact without worrying too much about what they mean. It's similar to algebra, in that we can make statement about `x + y = y + x` or `n + 0 = n` without worrying about whether x is measured in apples, lightyears, or anything at all.

goatlover 7y ago
Right, but if a future alien archaeologist were to dig up a thumb drive (preserved in a vault) after humans went extinct, what would indicate that it held bits?
- mattkrause 7y ago
  A 'bit' in information theory is just a unit of measurement, corresponding to the information conveyed by one yes-or-no question (where both answers were previously equally likely). Since it's a fairly natural unit, your alien archaeologist might even use it. However, if they had a weird taboo against binary, they could define a similar concept in other ways. In fact, we humans have too. Bletchley Park used "bans" or hartleys, which is the information carried by a single base-10 digit (without any a priori information). Nats use a base-e representation instead, which ties into statistical physics nicely. There's nothing stopping you from inventing a weird informational unit where 1 corresponds to "the answer to a seven way question were two of the answers are each twice as likely as the rest." So, suppose your alien digs up my thumbdrive. It examines it carefully and notices that each MOSFET is always in one of two states. That suffice to describe its information-carrying capacity in whatever system they use: bits, hartleys, nats, zorblaxen. A similar analysis of their own media would let them say "This thing is so sophisticated, it can hold the work of 10,000 scribes" or "This junk can barely contain a moment of our 16D videos." If your drive were packed with different types of media, they might even discover that some parts of the disk (e.g., ascii text) DON'T use all of the available capacity. None of this, of course, tells them what the data means. For that, you'd need to relate these data to some external source.
- antepodius 7y ago
  There's two different problems here: how you interpret arbitrary strings of bits and how you interpret arbitrary buts of matter as representing bits or not. Yes, figuring out/assuming that the bit cells in the thumb drive represented bits might be difficult. Then again, if you can look and see what the internal physical structure of this thing is, you might see a pattern of highs and lows. But sure, getting from an inefficient physical structure to a series of digits is a step. But once you're there, maths has things to say about the possible information content of strings of bits.

goatlover 7y ago

Right, but if a future alien archaeologist were to dig up a thumb drive (preserved in a vault) after humans went extinct, what would indicate that it held bits?

mattkrause 7y ago
A 'bit' in information theory is just a unit of measurement, corresponding to the information conveyed by one yes-or-no question (where both answers were previously equally likely). Since it's a fairly natural unit, your alien archaeologist might even use it. However, if they had a weird taboo against binary, they could define a similar concept in other ways. In fact, we humans have too. Bletchley Park used "bans" or hartleys, which is the information carried by a single base-10 digit (without any a priori information). Nats use a base-e representation instead, which ties into statistical physics nicely. There's nothing stopping you from inventing a weird informational unit where 1 corresponds to "the answer to a seven way question were two of the answers are each twice as likely as the rest." So, suppose your alien digs up my thumbdrive. It examines it carefully and notices that each MOSFET is always in one of two states. That suffice to describe its information-carrying capacity in whatever system they use: bits, hartleys, nats, zorblaxen. A similar analysis of their own media would let them say "This thing is so sophisticated, it can hold the work of 10,000 scribes" or "This junk can barely contain a moment of our 16D videos." If your drive were packed with different types of media, they might even discover that some parts of the disk (e.g., ascii text) DON'T use all of the available capacity. None of this, of course, tells them what the data means. For that, you'd need to relate these data to some external source.
antepodius 7y ago
There's two different problems here: how you interpret arbitrary strings of bits and how you interpret arbitrary buts of matter as representing bits or not. Yes, figuring out/assuming that the bit cells in the thumb drive represented bits might be difficult. Then again, if you can look and see what the internal physical structure of this thing is, you might see a pattern of highs and lows. But sure, getting from an inefficient physical structure to a series of digits is a step. But once you're there, maths has things to say about the possible information content of strings of bits.