3 ms·

It does select a function. The distribution over outputs conditioned on the input prompt is a different distribution, hence a different (random) function. It'

It does select a function. The distribution over outputs conditioned on the input prompt is a different distribution, hence a different (random) function. It's not important that there are no weight updates. The new function, the model conditioned on a few example input-output pairs, is good at producing new outputs for unseen inputs. That is what learning is. There is a reason that everyone working on LLMs is comfortable with this definition. It really does exactly match the definition that has been used for decades.

YeGoblynQueenne 4y ago
>> The distribution over outputs conditioned on the input prompt is a different distribution, hence a different (random) function. As I say in another comment (and if I understand what you mean correctly) that is not supported by observations. >> It's not important that there are no weight updates. The new function, the model conditioned on a few example input-output pairs, is good at producing new outputs for unseen inputs. To be blunt, I've seen as much evidence for that as I've seen evidence for the existence of the Sasquatch. The people who train those models don't even know what's in their datasets, let alone anyone being able to say that an "example" (i.e. a prompt) is "unseen".