3 ms·

It seems that the term "information" is being used in two different ways in this thread. The usual meaning of a bit of information is with regard to a probabili

It seems that the term "information" is being used in two different ways in this thread. The usual meaning of a bit of information is with regard to a probability distribution over messages which the user wants to send to a server. I don't think most people are used to thinking about bits in other contexts, so that's where the miscommunication is happening. Your interpretation, which I think is correct in this context, seems to be with regard to the entropy of a probability distribution over internet users, and the mutual information between that and the distribution over messages. The actual length of the message is irrelevant to the math once you fix the joint probability distribution. The argument others seem to be making is that the joint probability distribution is in fact not fixed, and that you can smear out the conditional probability over users given a message by shrinking the space of possible messages. In theory that seems possible, but I don't know enough to have any idea how well that would work in practice. If you shrank the message space to be small enough to be useful for this purpose, wouldn't that get in the way of usability?

enriquto 6y ago
> Your interpretation This is not "my" interpretation, it's the standard definition of information content in computer science, as given [1] by Shannon in 1948 and used by everybody since. [1] https://en.wikipedia.org/wiki/A_Mathematical_Theory_of_Communication https://en.wikipedia.org/wiki/A_Mathematical_Theory_of_Commu...
- vladTheInhaler 6y ago
  Obviously I'm familiar with the definition. If you didn't get that, you should probably read my comment again. It seems like you've somehow decided that people in this thread are arguing with you, but they're not. And anyways, it's a bit silly to get mad at people because they haven't studied information theory. Define the random variables M for message content and U for the identity of the user. The interpretation of "bits of information" that most people will have is H(M). The correct interpretation in this context is H(U). You seem to be confused about why people are talking about H(M) instead of H(U). But I think people correctly intuit that those aren't independent, so the mutual information I(U;M) = H(U) - H(U|M) is positive. And obviously if you change P(M), you will also change the amount of mutual information. That's why talking about sending fewer headers makes sense.

enriquto 6y ago

> Your interpretation This is not "my" interpretation, it's the standard definition of information content in computer science, as given [1] by Shannon in 1948 and used by everybody since. [1] https://en.wikipedia.org/wiki/A_Mathematical_Theory_of_Communication https://en.wikipedia.org/wiki/A_Mathematical_Theory_of_Commu...

vladTheInhaler 6y ago
Obviously I'm familiar with the definition. If you didn't get that, you should probably read my comment again. It seems like you've somehow decided that people in this thread are arguing with you, but they're not. And anyways, it's a bit silly to get mad at people because they haven't studied information theory. Define the random variables M for message content and U for the identity of the user. The interpretation of "bits of information" that most people will have is H(M). The correct interpretation in this context is H(U). You seem to be confused about why people are talking about H(M) instead of H(U). But I think people correctly intuit that those aren't independent, so the mutual information I(U;M) = H(U) - H(U|M) is positive. And obviously if you change P(M), you will also change the amount of mutual information. That's why talking about sending fewer headers makes sense.