4 ms·
> The assumption that most people make is that these models can answer questions or chat with you, but in reality all they can do is take some text you provide
by astrange 3y ago
> The assumption that most people make is that these models can answer questions or chat with you, but in reality all they can do is take some text you provide as input and guess what the next word (or more accurately, the next token) is going to be.
These two things cannot be compared or contrasted. It's very common to see people write something like "LLMs don't actually do <thing they obviously actually do>, they just do <dismissive description of the same thing>."
Typically, like here, the dismissive description just ignores the problem of why it manages to write complete novel sentences when it's only "guessing" subword tokens, why those sentences appear to be related to the question you asked, and why they are in the form of an answer to your question instead of another question (which is what base models would do).
- theshrike79 3y agoYeah, an LLM is not a Markov Chain. The only similarity is that they string words together with weighted possibilities. That's about it.
- astrange 3y agoWell, it is a Markov chain if you do greedy sampling, which 99% of the time you do. So the weird part is why it still works so well. If you do beam search, RAG, tool usage, etc then the whole system no longer is one.
- pottspotts 3y agoI came here to make this comment as well. This line of reasoning that LLMs "only predict" the next token is akin to saying humans can only think or speak one word at a time. Yes, we use one token/word at a time, but it is the aggregate thought that matters, regardless of what underlies it.
- winternewt 3y agoI think the mistake people make is assuming that "probability" is a simple concept. If there are 50K possible tokens and I don't have any other information, I could make a naive estimate that every token has equal probability and start generating text that is just gibberish. With the simple single-token Markov-chain example I would estimate probabilities based the previous token, and that probability estimate would be much better. If you use it for generating text it will look like something that is almost, but not quite, entirely unlike human speech. [1] The difference lies entirely in how accurately you model the world and what information you have available when estimating probabilities. Models like GPT4 happen to be very good at it because they encode a huge amount of knowledge about the world and take a lot of context into account when estimating the probability. That's not something to be taken lightly. [1] https://projects.haykranen.nl/markov/demo/ https://projects.haykranen.nl/markov/demo/
- XorNot 3y agoI am skeptical anyone saying this is making a mistake: it only ever really comes up when someone has specific priors they're wanting to litigate - best summarized by the timeless: you cannot make a man understand something when his paycheque depends on his not understanding it.
- lm28469 3y agoWhen the other camp is treating it like an oracle of truth and a sentient being it's hard to pick a side tbh.
- vasco 3y agoIf someone asks me "What is your name?" my reply is also simply just guessing what token would go well next in the full text of the conversation.
- miningape 3y agoNo you "know" what your name is and you retrieve that information. It's infinitely different to rolling a dice and picking a name based on what number comes up
- agumonkey 3y agoSometimes I'm tempted to think that my brain is as fuzzy if not more than an LLM
- realusername 3y agoI feel like I work in similar ways, if you ask my name there's a probability that I'll answer with the shorten version or the longer one pretty much randomly and there's no conscious effort about it. While the LLMs do have hallucinations, basic stuff like this will never trigger any. Where I feel the most differences are aren't in the token concept but rather on the deep reasoning (which we don't use as much in my opinion)
- astrange 3y agoThis is two different parts of the system. > No you "know" what your name is and you retrieve that information. The LLM has this (literally - most of it is a key value store). It outputs a probability for all possible next tokens. > It's infinitely different to rolling a dice and picking a name based on what number comes up The sampling algorithm running the LLM then does this part, and adds randomness to make it more "creative". So if you want factual information then don't be so random you skip over the right answer.
- poniko 3y agoIt's really not though, generally maybe but you also give it a split second to think about if it's a good time to lie, make a joke or maybe just not reveal your name.
- photon_lines 3y agoYeah - most of the online descriptions aren't even remotely accurate nor close to explaining how LLMs like ChatGPT actually work. They are not simple 'next-word predictors' and most of the online tutorials / info don't go into fine-tuning nor the intricate details of chain of thought reasoning (which I personally believe plays a huge role in ChatGPT's amazing performance). If you want my own detailed description you can find it here: https://photonlines.substack.com/p/intuitive-and-visual-guide-to-transformers https://photonlines.substack.com/p/intuitive-and-visual-guid...
- sk11001 3y agoI think too many people confuse the base model (which can be called a next token predictor) with the fine-tuned chat model which is specifically modified to carry a conversation, be helpful and be as factually correct as possible.
- 7sidedmarble 3y agoWhere can you read more on that matter of it answering your question rather than asking more?
- astrange 3y agoThat's called instruction tuning. https://arxiv.org/abs/2308.10792 https://arxiv.org/abs/2308.10792