6 ms·

If you assume a view in which syntax produces a predictable semantics, then yes. The number 123321 contains more information than 3. But natural languages do no

by derrida 15y ago

If you assume a view in which syntax produces a predictable semantics, then yes. The number 123321 contains more information than 3. But natural languages do not have a semantics produced by a syntax in an easily predictable way. I think it isn't a novel result, but for different reasons.

CWuestefeld 15y ago
The number 123321 contains more information than 3. This isn't true, or at least, it isn't true most relevant ways. Most obviously, if I need to specify which record in my database (of, say, a million records) is to be read, then "3" and "123321" contain precisely the same amount of information. But I think that even in more common cases: "how many birds are at the bird feeder?"; "how many gallons did it take to fill you gas tank?"; "what's you GPA?", the answer, regardless of its magnitude, conveys the same amount of information. Now, precision is a different story. "I'm going to take a couple of weeks vacation in July" contains less information than "I'm going to take 11 days of vacation in July". "My gas tank has a capacity of 16 gallons" contains less information than "my gas tank has a capacity of 15.9 gallons.".
- sesqu 15y ago
  There is interplay. "There are on the order of fifty birds at the bird feeder" is more informative than "there are on the order of zero birds at the bird feeder".
blauwbilgorgel 15y ago
But then again string A) 111111111111111 contains less information than string B) 48611 as string A can be written as a small program: 1*15. Like in the article it could be far more likely to have a 1 follow a 1 (like markov chaining probability) than another number, making the string A more probable and therefor containing less information. The overarching notion to me seems to be (Kolmogorov) complexity. The size of the smallest program that can reproduce the string/word/utterance denotes the complexity of that string/word/utterance. We use less complex words more often, not because they are shorter, but because their optimal compression ratio (not the number of morphenes, but their order and complexity) is shorter than other words. The program to produce it in our brains is smaller and simpler.
- ToastOpt 15y ago
  Suppose then I run a lottery, drawing 5 decimal digits. We have a special deal though -- if all 5 digits match, we draw again, up to 3 more times, and award a unique prize to the ticket holder for each result. We always sell precisely 10000 tickets (you don't get to pick your ticket). Last week, the results were 11111 11111 11111. This week, the results are 48611. In this case, would you suggest that last week's results convey less information? It does convey overall more facts than this week's results.

by derrida 15y ago

CWuestefeld 15y ago

The number 123321 contains more information than 3. This isn't true, or at least, it isn't true most relevant ways. Most obviously, if I need to specify which record in my database (of, say, a million records) is to be read, then "3" and "123321" contain precisely the same amount of information. But I think that even in more common cases: "how many birds are at the bird feeder?"; "how many gallons did it take to fill you gas tank?"; "what's you GPA?", the answer, regardless of its magnitude, conveys the same amount of information. Now, precision is a different story. "I'm going to take a couple of weeks vacation in July" contains less information than "I'm going to take 11 days of vacation in July". "My gas tank has a capacity of 16 gallons" contains less information than "my gas tank has a capacity of 15.9 gallons.".

sesqu 15y ago
There is interplay. "There are on the order of fifty birds at the bird feeder" is more informative than "there are on the order of zero birds at the bird feeder".

blauwbilgorgel 15y ago

But then again string A) 111111111111111 contains less information than string B) 48611 as string A can be written as a small program: 1*15. Like in the article it could be far more likely to have a 1 follow a 1 (like markov chaining probability) than another number, making the string A more probable and therefor containing less information. The overarching notion to me seems to be (Kolmogorov) complexity. The size of the smallest program that can reproduce the string/word/utterance denotes the complexity of that string/word/utterance. We use less complex words more often, not because they are shorter, but because their optimal compression ratio (not the number of morphenes, but their order and complexity) is shorter than other words. The program to produce it in our brains is smaller and simpler.

ToastOpt 15y ago
Suppose then I run a lottery, drawing 5 decimal digits. We have a special deal though -- if all 5 digits match, we draw again, up to 3 more times, and award a unique prize to the ticket holder for each result. We always sell precisely 10000 tickets (you don't get to pick your ticket). Last week, the results were 11111 11111 11111. This week, the results are 48611. In this case, would you suggest that last week's results convey less information? It does convey overall more facts than this week's results.