3 ms·

I'm looking for a definition of information that matches up with how we use the word in normal parlance. This definition has to strike some kind of medium betw

• thwarted 18y ago
  I'm not sure I follow. The definition of "information" I use everyday matches the "official" version -- I'm not sure how it could be different. What is "completely compressible"? Something can not be compressed beyond the shortest string that can represent it without losing information content. http://en.wikipedia.org/wiki/Information#Measuring_information_entropy http://en.wikipedia.org/wiki/Information#Measuring_informati...
  • yters 18y ago
    Shannon's information is a measure of the size of a set that an element is selected from, is that right? So a letter conveys log2(26) bits of information. That alone doesn't allow me to discriminate between the information content of a random string and an English sentence, since both supposedly contain the same amount of information by this bare metric. If I look at the occurrence of subsets of the string, then that would be a better discriminator: the random string's subsets should follow a normal distribution while the English string's subsets will be highly skewed. However, that doesn't work when I try to discriminate between an English string and a string generated by a simple algorithm, since the latter's subset distribution will also be highly skewed. What kind of metric discriminates the English sentence from either case? Am I making sense here? I haven't had any formal training in information theory, and my brain is kind of fried right now.