4 ms·

> For reasons that are not fully understood, text language models emit is less interesting than text emitted by humans. What's not understood? As you describe,

> For reasons that are not fully understood, text language models emit is less interesting than text emitted by humans. What's not understood? As you describe, every standard language model is statistically predicting the next word given the previous words, ad infinitum. GPT3 has more parameters to build a more precise "thought" vector, but the goal is still only to produce the next word.

drdeca 6y ago
Suppose you had an impossibly perfect conditional probability distribution over "text that a person would write". I realize there may be some issues with that being a well-defined concept. If you don't think it is well defined (and perhaps it isn't), please momentarily pretend that that is something that is/can-be well defined. In that case, for any property of a text, such as "the text gets ramble-y and basically doesn't have any real point", the probability that a randomly selected text produced by a person has the property, and the probability that a text sampled from that distribution, would be equal. Could text be sampled from such a distribution on a word by word (or token by token) basis? Yes. Let's see why: For any event (or property), there are countably many texts which satisfy the property. So, it can be expressed as a countable disjoint union of sets which each have as their only element one of the texts which have the property. So, the probability of a text sampled from the distribution having the property, is equal to the sum of the probabilities of each of the texts which have the property. The probability of each text is equal to the probability of "the next token is [the last word of the text]" given [all the tokens before the last one], multiplied by the probability of the text [all of the tokens before the last one]. And then recurse on that, in order to break the probability of each text down into products of conditional probabilities of (next token | previous tokens) . Hm, actually, I think there's an issue with what I said. It would work if all texts people wrote had the same length, or if all of them continued indefinitely, but that's not the case. Maybe that can be patched by including a conditional probability for an "end of text" token. Actually, yes, that seems likely. Say that the probability of the text is the probability of (that is all of the text | [the text] is some initial segment of the text), multiplied by the probability of ([the text] is some initial segment of the text). and then the probability of ([an initial segment] is some initial segment of the text) is the probability of ([the last token of the initial segment] is the next token in an initial segment of the text | [the tokens before the last one of the segment] is an initial segment of the text ) times the probability of ([the tokens before the last one of the segment] is an initial segment of the text). Ok, yeah. So, in theory, if you allow arbitrary conditional probabilities for the next token (or an end-of-text token) conditioned on all the previous tokens, you can produce any probability distribution over texts. Therefore, "predicting the next word given the prior words" isn't any limitation on what probability distributions over text can be produced. (though perhaps it could drastically increase the cost of producing certain probability distributions, and perhaps in such a way that makes it practically infeasible to produce in such a way.)

drdeca 6y ago

Suppose you had an impossibly perfect conditional probability distribution over "text that a person would write". I realize there may be some issues with that being a well-defined concept. If you don't think it is well defined (and perhaps it isn't), please momentarily pretend that that is something that is/can-be well defined. In that case, for any property of a text, such as "the text gets ramble-y and basically doesn't have any real point", the probability that a randomly selected text produced by a person has the property, and the probability that a text sampled from that distribution, would be equal. Could text be sampled from such a distribution on a word by word (or token by token) basis? Yes. Let's see why: For any event (or property), there are countably many texts which satisfy the property. So, it can be expressed as a countable disjoint union of sets which each have as their only element one of the texts which have the property. So, the probability of a text sampled from the distribution having the property, is equal to the sum of the probabilities of each of the texts which have the property. The probability of each text is equal to the probability of "the next token is [the last word of the text]" given [all the tokens before the last one], multiplied by the probability of the text [all of the tokens before the last one]. And then recurse on that, in order to break the probability of each text down into products of conditional probabilities of (next token | previous tokens) . Hm, actually, I think there's an issue with what I said. It would work if all texts people wrote had the same length, or if all of them continued indefinitely, but that's not the case. Maybe that can be patched by including a conditional probability for an "end of text" token. Actually, yes, that seems likely. Say that the probability of the text is the probability of (that is all of the text | [the text] is some initial segment of the text), multiplied by the probability of ([the text] is some initial segment of the text). and then the probability of ([an initial segment] is some initial segment of the text) is the probability of ([the last token of the initial segment] is the next token in an initial segment of the text | [the tokens before the last one of the segment] is an initial segment of the text ) times the probability of ([the tokens before the last one of the segment] is an initial segment of the text). Ok, yeah. So, in theory, if you allow arbitrary conditional probabilities for the next token (or an end-of-text token) conditioned on all the previous tokens, you can produce any probability distribution over texts. Therefore, "predicting the next word given the prior words" isn't any limitation on what probability distributions over text can be produced. (though perhaps it could drastically increase the cost of producing certain probability distributions, and perhaps in such a way that makes it practically infeasible to produce in such a way.)