4 ms·
My rule of thumb is to take every single LLM prompt and just imagine that it's prefixed with: "Computer, focus on generating output that resembles the words pe
by Terr_ 3y ago
My rule of thumb is to take every single LLM prompt and just imagine that it's prefixed with:
"Computer, focus on generating output that resembles the words people in the past used after they were given the following words..."
- jmprspret 3y agoThat is accurate to what they do. I think others need to imagine this as well. Far too many nontechnical people seem to treat them as some kind of Oracle.
- inopinatus 3y agoCorrect. You must perceive them as plausibility engines. The unstated hypothesis is that plausibility of output may converge towards correctness of output with increasing scale and sophistication. This hypothesis remains very far from proven.
- __loam 3y agoI don't think it's that hard to understand what the hell is going on with LLMs under the hood. Ultimately it's a weighted sample of the training data. It has a relationship with reality insofar as one exists within the training data. HFRL makes it easier to believe something crazy is happening because the output is being weighted towards something that's believable to us.
- dTal 3y agoDepending on what you mean by "weighted sample", that's either trivially true (the network is of course a function of its training data and nothing else) or trivially false (the network generalizes over the training data and has not memorized it). It is not a good intuition pump for why an LLM can hold up one end of a conversation, or follow novel instructions - it is not reading from a script, nor regurgitating chopped up pieces of text like a Markov chain. It is doing something very clever in a way that is not obvious. >It has a relationship with reality insofar as one exists within the training data This is true of anything that learns.
- bamboozled 3y agoI also think you get the best results when thinking about using them this way too, any other way of using them seems to end in disappointment.
- panarky 3y agoYour understanding of how LLMs work is overly simplistic and incomplete. Yes, doing probabilistic next-word prediction plays a role in how LLMs generate text output, but that's not the whole story. LLMs "understand" (to a degree): They develop complex internal representations of concepts they've been trained on. This isn't just about word association; they develop an understanding of the relationships between objects, actions, and ideas. They can reasoning, not just mimic: LLMs can perform logical reasoning, using their internal knowledge base to solve problems or answer questions. This might involve following multi-step instructions, drawing inferences from information provided, or adapting to new prompts in a way that requires a degree of abstract thinking. Beyond simple probabilities: Yes, LLMs do consider the probability of certain word sequences, but their output is far more sophisticated than just picking the most likely next word. They weigh context, concepts, relationships, nuance, logic, and even the unstated but inferred purpose of the user when generating responses.
- seizethecheese 3y agoI feel like this is the midwit meme response and the idiot and genius are just saying “it’s next token probabilistic geberation”
- xcv123 3y agoA typical mid-wit response is to say it's just a Markov chain doing naive next token prediction without any semantic model. That's not how deep learning works. The LLM neural network contains a semantic model and it performs some type of reasoning over that model. The idiot and the genius both can see that ChatGPT has some reasoning capability. https://www.cs.toronto.edu/~hinton/absps/AIJmapping.pdf https://www.cs.toronto.edu/~hinton/absps/AIJmapping.pdf "This 1990 paper demonstrated how neural networks could learn to represent and reason about part-whole hierarchical relationships, using family trees as the example domain. By training on examples of family relations like parent-child and grandparent-grandchild, the neural network was able to capture the underlying logical patterns and reason about new family tree instances not seen during training. This seminal work highlighted that neural networks can go beyond just memorizing training examples, and instead learn abstract representations that enable reasoning and generalization"