3 ms·
But then again string A) 111111111111111 contains less information than string B) 48611 as string A can be written as a small program: 1*15. Like in the articl
by blauwbilgorgel 15y ago
But then again string A) 111111111111111 contains less information than string B) 48611 as string A can be written as a small program: 1*15.
Like in the article it could be far more likely to have a 1 follow a 1 (like markov chaining probability) than another number, making the string A more probable and therefor containing less information.
The overarching notion to me seems to be (Kolmogorov) complexity. The size of the smallest program that can reproduce the string/word/utterance denotes the complexity of that string/word/utterance.
We use less complex words more often, not because they are shorter, but because their optimal compression ratio (not the number of morphenes, but their order and complexity) is shorter than other words. The program to produce it in our brains is smaller and simpler.
- ToastOpt 15y agoSuppose then I run a lottery, drawing 5 decimal digits. We have a special deal though -- if all 5 digits match, we draw again, up to 3 more times, and award a unique prize to the ticket holder for each result. We always sell precisely 10000 tickets (you don't get to pick your ticket). Last week, the results were 11111 11111 11111. This week, the results are 48611. In this case, would you suggest that last week's results convey less information? It does convey overall more facts than this week's results.