18 ms·

If a text corpus correlates with human knowledge (your "presupposes a certain model of the world"), then it necessarily encodes the portions of the knowledge it

If a text corpus correlates with human knowledge (your "presupposes a certain model of the world"), then it necessarily encodes the portions of the knowledge it correlates with. That is, correlations drive the search of model parameters that explain the correlations. The stronger or more numerous the correlations, the more accurate a model can be computed. But the search space informed by this network of correlations is much more restricted than a naive brute force search, thus the correlations provide information about, i.e. encodes, the correct model.

tsimionescu 6y ago
Then I used the wrong words. My point is that human text is essentially encrypted using a particular model of the world as the encryption/decryption key. You can't derive the key from the cipher text. For example, if I give you the text 'the car is red', there is no watly to imagine the meaning of this phrase (in any sense of the word 'meaning') unless you have some understanding of what cars are, what red is, and what it means for a car to be red (which is somewhat different from what it means for an apple to be red, for example). It is interesting that with enough statistical analysis you can begin to decipher certain parts of the true message without knowing the cipher key. But this only works for those parts where you have a large corpus of text talking about them, and there are many common human needs or experiences that do not have such a corpus. It is also very easy to capture answer/response pairs in the corpus and repeat them, seeming to have high accuracy, but in fact failing to generalize more than meets the eye. For example, with enough compute power, you could build a machine that, for any query, would find the most similar such query in all of its training data, and respond with the continuation of that query (which could be an answer or a follow-up question). This could be quite impressive in conversation, while doing nothing to elucidate anything about human intelligence, and being wholly unable to adapt to any novel situation.
- hackinthebochs 6y ago
  Your encryption analogy doesn't really work either. Encrypted text encodes the key used to encrypt it. This is why it takes 2^N computational steps to recover the plain text, where N is the key size rather than where N is the text size. It's just that the computational cost of decrypting with sufficiently large keys is intractable. But human languages have much more exposed regularity than ciphertext from good encryption algorithms, so the computational cost is much lower. The "key", i.e. world model, in case of human languages is plausibly much bigger, but since human languages have significant exposed regularity, it is still tractable to compute, with the trade off that you need much more text to do it.

tsimionescu 6y ago

Then I used the wrong words. My point is that human text is essentially encrypted using a particular model of the world as the encryption/decryption key. You can't derive the key from the cipher text. For example, if I give you the text 'the car is red', there is no watly to imagine the meaning of this phrase (in any sense of the word 'meaning') unless you have some understanding of what cars are, what red is, and what it means for a car to be red (which is somewhat different from what it means for an apple to be red, for example). It is interesting that with enough statistical analysis you can begin to decipher certain parts of the true message without knowing the cipher key. But this only works for those parts where you have a large corpus of text talking about them, and there are many common human needs or experiences that do not have such a corpus. It is also very easy to capture answer/response pairs in the corpus and repeat them, seeming to have high accuracy, but in fact failing to generalize more than meets the eye. For example, with enough compute power, you could build a machine that, for any query, would find the most similar such query in all of its training data, and respond with the continuation of that query (which could be an answer or a follow-up question). This could be quite impressive in conversation, while doing nothing to elucidate anything about human intelligence, and being wholly unable to adapt to any novel situation.

hackinthebochs 6y ago
Your encryption analogy doesn't really work either. Encrypted text encodes the key used to encrypt it. This is why it takes 2^N computational steps to recover the plain text, where N is the key size rather than where N is the text size. It's just that the computational cost of decrypting with sufficiently large keys is intractable. But human languages have much more exposed regularity than ciphertext from good encryption algorithms, so the computational cost is much lower. The "key", i.e. world model, in case of human languages is plausibly much bigger, but since human languages have significant exposed regularity, it is still tractable to compute, with the trade off that you need much more text to do it.