3 ms·
LLMs will predict the most likely output to the prompt input - does this mean that a hypothetical perfect LLM would produce the most generic and widely-acceptab
by Fripplebubby 3y ago
LLMs will predict the most likely output to the prompt input - does this mean that a hypothetical perfect LLM would produce the most generic and widely-acceptable response to any input, the "lowest common denominator" output?
The author seems to think so, and indeed, with today's GPTs, that is often what you might get back. But I think there's a lot more going on here - with natural language, a lot of the training effort goes into producing natural language output which conforms to the formal rules of language (it is intelligible, syntactically correct or roughly so, follows grammar, and semantically "about" what it is supposed to be about). But the interesting thing about natural language is that there are infinitely many correct responses which are valid formal responses to the prompt, both ones that introduce new or rare creative possibilities and ones that tread the same ground as a wikipedia article, for example (which is usually intended to be very neutral). So an LLM can produce both types of responses - formally correct and boring, formally correct and interesting - and you can see this today by offering "mix-ins" to the prompt like, "Tell me about Saudi Arabia's ruling family, in the style of Peter Griffin". Of course, it won't select to use the style of Peter Griffin without that prompt, but I think you can imagine a "weirdo" LLM that is trained with some positive response toward creative or interesting output too.
- Fripplebubby 3y agoI don't mean to imply that creativity is something that can be quantified objectively (used as the objective function, even), but it is widely understood that LLMs can produce varied output and be given particular biases or personalities
- hansvm 3y agoLLMs don't generally predict the most likely output, except when people go in to tweak temperature and other settings to make them more predictable for certain more mechanical tasks. The whole point of an LLM is that its output is calibrated; it produces a text with the same conditional probability you would expect seeing that text in the wild (give or take that we still don't understand why neural networks generalize).
- Fripplebubby 3y agoWould the output with the same conditional probability not also be the most likely in a conventional sense? I'm not understanding the distinction, but I'm not an expert so this might be a subtlety I just don't grasp
- hansvm 3y agoConditional probabilities are probabilistic. If you ask again you might get a different answer. The most likely answer is a single sample. It often isn't actually the result you want, it often has a much lower probability of occurring than your intuition would suggest, and even if it's correct once it's often not something you would appreciate appearing repeatedly if you re-asked the same question. I can add more details/comparisons/nuance/explanations/... if you're curious why that matters, but hopefully the distinction is clear now?
- Fripplebubby 3y agoI think that helped, thanks!