7 ms·
How LLMs Work, Explained Without Math
- thefz 2y ago> On the other side, given the propensity of LLMs to hallucinate, I wouldn't trust any workflow in which the LLM produces output that goes straight to end users without verification by a human. Yep. Nice article, though!
- astrange 2y ago> The assumption that most people make is that these models can answer questions or chat with you, but in reality all they can do is take some text you provide as input and guess what the next word (or more accurately, the next token) is going to be. These two things cannot be compared or contrasted. It's very common to see people write something like "LLMs don't actually do <thing they obviously actually do>, they just do <dismissive description of the same thing>." Typically, like here, the dismissive description just ignores the problem of why it manages to write complete novel sentences when it's only "guessing" subword tokens, why those sentences appear to be related to the question you asked, and why they are in the form of an answer to your question instead of another question (which is what base models would do).
- theshrike79 2y agoYeah, an LLM is not a Markov Chain. The only similarity is that they string words together with weighted possibilities. That's about it.
- astrange 2y agoWell, it is a Markov chain if you do greedy sampling, which 99% of the time you do. So the weird part is why it still works so well. If you do beam search, RAG, tool usage, etc then the whole system no longer is one.
- pottspotts 2y agoI came here to make this comment as well. This line of reasoning that LLMs "only predict" the next token is akin to saying humans can only think or speak one word at a time. Yes, we use one token/word at a time, but it is the aggregate thought that matters, regardless of what underlies it.
- winternewt 2y agoI think the mistake people make is assuming that "probability" is a simple concept. If there are 50K possible tokens and I don't have any other information, I could make a naive estimate that every token has equal probability and start generating text that is just gibberish. With the simple single-token Markov-chain example I would estimate probabilities based the previous token, and that probability estimate would be much better. If you use it for generating text it will look like something that is almost, but not quite, entirely unlike human speech. [1] The difference lies entirely in how accurately you model the world and what information you have available when estimating probabilities. Models like GPT4 happen to be very good at it because they encode a huge amount of knowledge about the world and take a lot of context into account when estimating the probability. That's not something to be taken lightly. [1] https://projects.haykranen.nl/markov/demo/ https://projects.haykranen.nl/markov/demo/
- XorNot 2y agoI am skeptical anyone saying this is making a mistake: it only ever really comes up when someone has specific priors they're wanting to litigate - best summarized by the timeless: you cannot make a man understand something when his paycheque depends on his not understanding it.
- lm28469 2y agoWhen the other camp is treating it like an oracle of truth and a sentient being it's hard to pick a side tbh.
- vasco 2y agoIf someone asks me "What is your name?" my reply is also simply just guessing what token would go well next in the full text of the conversation.
- miningape 2y agoNo you "know" what your name is and you retrieve that information. It's infinitely different to rolling a dice and picking a name based on what number comes up
- agumonkey 2y agoSometimes I'm tempted to think that my brain is as fuzzy if not more than an LLM
- realusername 2y agoI feel like I work in similar ways, if you ask my name there's a probability that I'll answer with the shorten version or the longer one pretty much randomly and there's no conscious effort about it. While the LLMs do have hallucinations, basic stuff like this will never trigger any. Where I feel the most differences are aren't in the token concept but rather on the deep reasoning (which we don't use as much in my opinion)
- astrange 2y agoThis is two different parts of the system. > No you "know" what your name is and you retrieve that information. The LLM has this (literally - most of it is a key value store). It outputs a probability for all possible next tokens. > It's infinitely different to rolling a dice and picking a name based on what number comes up The sampling algorithm running the LLM then does this part, and adds randomness to make it more "creative". So if you want factual information then don't be so random you skip over the right answer.
- poniko 2y agoIt's really not though, generally maybe but you also give it a split second to think about if it's a good time to lie, make a joke or maybe just not reveal your name.
- photon_lines 2y agoYeah - most of the online descriptions aren't even remotely accurate nor close to explaining how LLMs like ChatGPT actually work. They are not simple 'next-word predictors' and most of the online tutorials / info don't go into fine-tuning nor the intricate details of chain of thought reasoning (which I personally believe plays a huge role in ChatGPT's amazing performance). If you want my own detailed description you can find it here: https://photonlines.substack.com/p/intuitive-and-visual-guide-to-transformers https://photonlines.substack.com/p/intuitive-and-visual-guid...
- sk11001 2y agoI think too many people confuse the base model (which can be called a next token predictor) with the fine-tuned chat model which is specifically modified to carry a conversation, be helpful and be as factually correct as possible.
- 7sidedmarble 2y agoWhere can you read more on that matter of it answering your question rather than asking more?
- astrange 2y agoThat's called instruction tuning. https://arxiv.org/abs/2308.10792 https://arxiv.org/abs/2308.10792
- Cyphase 2y ago[flagged]
- fredoliveira 2y agoThis type of comment blows my mind and frankly, I expect more from people who browse HN. Is this supposed to be a dig at the author? Since when is writing a book about another technology somehow indicative of a lack of expertise in anything else? If anything, the generality just means the guy is smart and able to apply his knowledge across multiple subject matters.
- Cyphase 2y agoI guess I knew I was taking a risk with that comment, but I was thinking more of it being low value; I didn't even consider that it might be taken this way. It wasn't a dig of any sort. It was just a late-night quote-unquote "interesting" observation from a Python fan, with simonw being known for Django and LLM research. I probably should have added something to make that clearer.
- fredoliveira 2y agoFair, and I appreciate the explanation. I do think this provides value. Perhaps not to everyone here, or people who have been working on LLMs for a while. But to people who think they have no ability to grok (pun not intended) LLMs because they don't have the mathematical chops? I think it'll be valuable for them.
- Cyphase 2y agoWow, there's really a kink in the wire somewhere between you and I. :) > I guess I knew I was taking a risk with that comment, but I was thinking more of it being low value; I didn't even consider that it might be taken this way. > ... I was thinking more of it being low value ... "It" being my comment; I was taking a risk because my intention with the comment was just a relatively bland observation. A low value _comment_. Again, no digs about the article whatsoever, at all, 0.000%. You are completely misunderstanding my intention. No hard feelings at all, just want to clarify that for the record. :) My comment was just a low value, passing observation, basically equivalent to if this article had been written by Sarah Wexler (made up name) and I had said, "Hmm, another person with the initials S.W. explaining LLMs..".
- mjburgess 2y agoIf commenters wish to know what is not, "guessing the next word", let me outline it. Compare, "I like what you were wearing", "Pass me the salt", and "Have you been to London recently?" as generated by an LLM and as spoken by a person. What is the reason each piece of text (in a whatsapp chat, say) is provided? When the LLM generates each word it does so because it is, on average, the most common word in a corpus of text on which it was trained: "wearing" follows, "I like what you were" because most people who were having these conversations, captured in the training data, were talking about clothes. When a person types those words on a keyboard, the following are the causes: the speaker's mental states of recollection, preference, taste; the speaker's affective/attachement states with respect to their friend; the speaker's habitation into social cues; the speaker's imagining through recall what their friend was wearing; the speaker's ability to abstract from their memories into identifying clothing; and so on. Indeed, the cause of a person speaking is so vastly different to generating a word based on a historical frequency, that to suppose these are related seems incomprehensible. The only reason the illusion of similarity is effective is because the training data is a text-based observation of the causal process in people: the training data is distributed by people talking (and so on). Insofar as you cannot just replay variations on these prior conversations, the LLM will fail and expose itself as actually insensitive to any of these things. I'd encourage credulous fans of AI not to dehumanize themselves and others by the supposition that they speak because they are selecting an optimal word from a dictionary based on all prior conversations they were a part of. You aren't doing that.
- jameshart 2y agoLLMs don’t ’replay variations on prior conversations’ though. Predicting the probable token in a conversation requires predicting the probable subject of the conversation, predicting the interlocutors’ relationship and manner of speaking to one another, predicting the state of recollection, preference and taste of the speaker, predicting the speaker’s mental model… If the LLM isn’t predicting all of those things then it will produce poor predictions of the next word; doing it well - and humans tend to agree that in a vast array of cases state of the art LLMs do predict tokens very well - requires that prediction model to predict all that context as well.
- l5870uoo9y 2y agoAre there any open source implementation of neural network "functions"? And the layering, transformers and attention mechanisms.
- barfbagginus 2y agoHere's a walkthrough nlp.seas.harvard.edu/annotated Here's a cool animated transformer, also open source prvnsmpth.github.io/animated Here's the "attention is all you need paper" with links to open source implementations paperswithcode.com/paper/attention-is-all
- mft_ 2y agoHow does this concept explain (for example) an LLM’s ability to provide a precis of an article? Or to compare two blocks of text and highlight differences? Or to take an existing block of code and find and correct an error?
- lnenad 2y agoExactly what I am wondering, these are the complex things that "Oh it just dumps out tokens" explanations don't explain.
- amanzi 2y agoStephen Wolfram has a great post on this: https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-doing-and-why-does-it-work/ https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-... It's really long! But worth the read.
- pietmichal 2y agoThis was such a nice primer that inspired me to give Karpathy's series another try. Loved the explanation!
- z7 2y ago"I'll begin by clearing a big misunderstanding people have regarding how Large Language Models work. The assumption that most people make is that these models can answer questions or chat with you, but in reality all they can do is take some text you provide as input and guess what the next word (or more accurately, the next token) is going to be." What separates this from the following: "I'll begin by clearing a big misunderstanding people have regarding how the human brain works. The assumption that most people make is that the brain can think, reason, and understand language, but in reality all it can do is process electrical and chemical signals."
- seydor 2y agoyeah i think we are at a point of reflection in philosophy of mind, where we ponder if that our "thinking", our communication and dialogues are mechanistic continuations taking place serially in one or more brains
- z7 2y agoWhether brains or LLMs, claiming that "in reality all they can do" is X seems dubious, given that a system can have properties that its constituents do not have on their own. As an explanation of the emerging complexity of what "they can do" it is fundamentally unsatisfying.
- deleted 2y ago[deleted]
- alabhyajindal 2y agoGreat read, thanks for sharing!