3 ms·
Yes but grammatically correct sequences of words have much less entropy than that. I guess much much less.
by pfortuny 13y ago
Yes but grammatically correct sequences of words have much less entropy than that. I guess much much less.
- sp332 13y agoAs a rule of thumb, English text has about one bit per character of entropy. [0, 1] Since we're going with averages, let's say 5 letters + a space for each word. So you need a 7- or 8-word sentence, with normal capitalization and punctuation, to get 42 bits of entropy. And of course it shouldn't be a well-known phrase like "I've got a bad feeling about this!" [0] The original http://www.princeton.edu/~wbialek/rome/refs/shannon_51.pdf http://www.princeton.edu/~wbialek/rome/refs/shannon_51.pdf [1] and some evidence that it's still correct http://en.wikipedia.org/wiki/Hutter_Prize http://en.wikipedia.org/wiki/Hutter_Prize
- pfortuny 13y agoThanks, great info.