4 ms·

I understand the Log2 concept of people able to narrow down something via binary search, but I have a question: don't the "facts" about a person have to divide

I understand the Log2 concept of people able to narrow down something via binary search, but I have a question: don't the "facts" about a person have to divide the remaining population in half (or into smaller chunks)? For instance if you know "Frank" doesn't wear a Rolex, that would not rule out very many people. So statistically, it would probably be better to know if Frank has red hair, as that could rule out a lot more people. Also, let's say you have it narrowed down to four people, but the last bit of information is common to all of them. You now have to get another bit, and possibly another, correct? EDIT: Felt like I didn't express my main point well enough: while you can certainly narrow down people with "bits" of information, information is most of the time not just 1 or 0 and can be fuzzy (or too common) to be useful in a binary search, although with the right bits of information it can of course be fruitful. I'm really interested by this concept and also curious as to if anyone is employing it on a mass scale.

blake8086 15y ago
You can learn part of a bit from a fact that doesn't divide the population neatly in half: http://en.wikipedia.org/wiki/Entropy_(information_theory)#Example http://en.wikipedia.org/wiki/Entropy_(information_theory)#Ex...
jerf 15y ago
"Also, let's say you have it narrowed down to four people, but the last bit of information is common to all of them." The definition of a bit is something which removes half the possibilities. If you have 4 people and acquire a "bit" of information that breaks them into two categories, one with 4 people and one with 0 people, you, by definition, in fact have 0 bits. Fractional bits are not only possible, they are by far the common case. With a lg2 in the definition of the bit, it's pretty uncommon to have integral bits. Critical insight: What we call a "bit" in a computer and a "bit" in information theory are related but not the same thing. You can't have a fraction of a bit stored in your computer's RAM, the words are meaningless. It is best to simply flush your idea of what you think a bit is and start over again from scratch when studying information theory, then when you are comfortable with it the connections will become obvious. Starting from the RAM side is actively harmful.
- orthecreedence 15y ago
  This makes sense, thanks for explaining the semantics. So basically, you're confirming that you'd need 33 "bits" of information, which doesn't necessarily mean you can use 33 pieces of information because a bit is a lot more specific than a one-off piece of information.
  - pohl 15y ago
    33 bits of information can amount to fewer than 33 pieces of information. Quoting from the article: "...knowing your hometown gives me 16 bits of entropy about you". Edit: a relevant anecdote: when I was in high school, a friend went as an exchange student to Costa Rica. As an experiment, I had her send me a blank postcard with nothing on it but "pohl 68320 USA". It arrived in my post office box without a hitch.
    - deleted 15y ago
      [deleted]
    - repsilat 15y ago
      Conversely, though, lots of pieces of data can end up not adding a lot of information, because a lot of data is correlated. Consider the following three facts: 1. A person is Chinese (~2 bits of information) 2. A person lives in Asia (<1 bit of information) 3. A person lives in the USA (4 or 5 bits of information) Now consider combinations of these facts. 1&2 together probably doesn't give you much more than two bits of information still. Conversely, though, 1&3 together give you 10 or 11 bits of information - significantly more than the information of those facts added together. This is because facts 1&2 have a strong positive correlation, and facts 1&3 have a strong negative correlation.
deleted 15y ago
[deleted]

blake8086 15y ago

You can learn part of a bit from a fact that doesn't divide the population neatly in half: http://en.wikipedia.org/wiki/Entropy_(information_theory)#Example http://en.wikipedia.org/wiki/Entropy_(information_theory)#Ex...

jerf 15y ago

"Also, let's say you have it narrowed down to four people, but the last bit of information is common to all of them." The definition of a bit is something which removes half the possibilities. If you have 4 people and acquire a "bit" of information that breaks them into two categories, one with 4 people and one with 0 people, you, by definition, in fact have 0 bits. Fractional bits are not only possible, they are by far the common case. With a lg2 in the definition of the bit, it's pretty uncommon to have integral bits. Critical insight: What we call a "bit" in a computer and a "bit" in information theory are related but not the same thing. You can't have a fraction of a bit stored in your computer's RAM, the words are meaningless. It is best to simply flush your idea of what you think a bit is and start over again from scratch when studying information theory, then when you are comfortable with it the connections will become obvious. Starting from the RAM side is actively harmful.

orthecreedence 15y ago
This makes sense, thanks for explaining the semantics. So basically, you're confirming that you'd need 33 "bits" of information, which doesn't necessarily mean you can use 33 pieces of information because a bit is a lot more specific than a one-off piece of information.
- pohl 15y ago
  33 bits of information can amount to fewer than 33 pieces of information. Quoting from the article: "...knowing your hometown gives me 16 bits of entropy about you". Edit: a relevant anecdote: when I was in high school, a friend went as an exchange student to Costa Rica. As an experiment, I had her send me a blank postcard with nothing on it but "pohl 68320 USA". It arrived in my post office box without a hitch.
  - deleted 15y ago
    [deleted]
  - repsilat 15y ago
    Conversely, though, lots of pieces of data can end up not adding a lot of information, because a lot of data is correlated. Consider the following three facts: 1. A person is Chinese (~2 bits of information) 2. A person lives in Asia (<1 bit of information) 3. A person lives in the USA (4 or 5 bits of information) Now consider combinations of these facts. 1&2 together probably doesn't give you much more than two bits of information still. Conversely, though, 1&3 together give you 10 or 11 bits of information - significantly more than the information of those facts added together. This is because facts 1&2 have a strong positive correlation, and facts 1&3 have a strong negative correlation.

deleted 15y ago

[deleted]