4 ms·
Doesn't the LLM still output the next most probable token (modulus heat/desired variation) according to its model? Training shifts the probability - but doesn't
by e12e 2mo ago
Doesn't the LLM still output the next most probable token (modulus heat/desired variation) according to its model? Training shifts the probability - but doesn't change the fact that the output is a sampling based on input and the model?
- josu 2mo agoThe combination of the most probable tokens doesn't necessarily have to be in the training set, thus creating something completely novel.
- DiabloD3 2mo agoThat is called a hallucination.
- Gander5739 2mo agoOnly if it's incorrect.
- deleted 2mo ago[deleted]
- josu 2mo agoYes, like the Jacobian counterexample.
- e12e 2mo agoIf you shuffle a deck of cards, you likely have a novel sequence of cards. I'm not sure how that is relevant?
- josu 2mo agoI was responding to this "merely produces syntactically likely combinations of the training data".
- bob001 2mo agoSo do humans more or less? The only counterpoints I've heard are religious or unmeasured quantum brain something.
- e12e 2mo agoI think there are quite fundamental differences to the LLM model equivalent to an adult human mind and current LLMs (even accepting an LLM can model a human brain, which I don't think is a given - but I also don't think it's impossible). The details matter (training set, full context). I doubt our current LLMs knows what smelling cut grass on a wet morning feels like, or how the stomach flutters on a first kiss - even if they have "read" tens to hundreds of attempts at describing such things. You're essentially suggesting we play Turing's Guessing Game; if your LLM can guess my response to any prompt - then it can be said to have modeled how I would reply to those prompts. Expand the prompts far enough, and you could reasonably argue the LLM models me.
- bob001 2mo ago>The details matter (training set, full context). But you said: >Training shifts the probability - but doesn't change the fact that the output is a sampling based on input and the model? So which is it? >You're essentially suggesting we play Turing's Guessing Game; if your LLM can guess my response to any prompt - then it can be said to have modeled how I would reply to those prompts. Expand the prompts far enough, and you could reasonably argue the LLM models me. I'm merely noting that your argument that "training+input=>output is the problem" applies to humans. There's other arguments for and against LLMs however I'm simply pointing out that your argument isn't a good one.
- e12e 2mo ago> I'm merely noting that your argument that "training+input=>output is the problem" applies to humans. I'm not sure I ever said that it was a problem. I also never meant to say that sampling across the model (formed by training) based on input couldn't produce novel token sequences. As for human vs LLM - my argument would be that while trained on a large corpus, the models are poor in experience and "sensory input" - their training so different from growing up - that even if we could model humans as LLMs - the gap between current LLMs and "human" LLMs remain a wide one. More to the point - I believe the current generation of LLMs are way too sensitive to input (prompt, system prompt, context/RAG) - precisely because they're still too narrowly trained, and also aligned to weigh input heavily. What we have today are models that work well with language (generate, transform, translate) - not so great on complex tasks - because they're still generating next probable token.
- semiquaver 2mo agoIt’s not sampling anything though. At each inference step the fundamental output of the process can be interpreted as a probability for each possible token how likely it is. Sampling would imply otherwise but the probabilities are the output.