6 ms·
Do I understand it correctly, that an LLMs are neural networks which only ever output a single "token", which is a short string of a few chars? And then the who
by TekMol 3y ago
Do I understand it correctly, that an LLMs are neural networks which only ever output a single "token", which is a short string of a few chars? And then the whole input plus that output is fed back into the NN to produce the next token?
So if you ask ChatGPT "Describe Berlin", what happens is that the NN is called 6 times with these inputs:
Input: Describe Berlin.
Outpu: Berlin
Input: Describe Berlin. Berlin
Outpu: is
Input: Describe Berlin. Berlin is
Outpu: a
Input: Describe Berlin. Berlin is a
Outpu: nice
Input: Describe Berlin. Berlin is a nice
Outpu: city
Input: Describe Berlin. Berlin is a nice city
Outpu: .
ChatGPT's answer:
Berlin is a nice city.
Is that how LLMs work?
- going_ham 3y agoThis is a great example to show the underlying mechanism!
- TekMol 3y agoGreat, thanks for the clarification. And how does the NN represent the token at the output layer? Is it a binary representation of the token number? Or does it have a neuron for each token it knows and ChatGPT takes the most activated neuron as the answer?
- zwaps 3y agoBasically the latter
- weinzierl 3y agoThe tokenization algorithms I encountered all had around 50000 tokens, which fits nicely into (and makes good use of) a 16-bit number. Is this just a coincidence or does it have advantages for the token to be a 16-bit representable number?
- danuker 3y agoI suspect it being 16 bit instead of 32 bit means more of them can get packed more tightly. Some instructions can operate on them in parallel. But I personally think it's a coincidence, and it just so happens that 50k tokens are enough for the level of complexity the models have right now.
- sebzim4500 3y agoProbably a coincidence. The GPT-4 and GPT-3.5 tokenizer has 100k tokens.
- mirekrusin 3y agoTokens are integers that map to text tokens. Tokens are part of words, approx 4 characters or 75% of word. It gives a list of tokens with their probabilities on output. It's a short list with highest probabilities. Temperature controls which tokens to pick - usually 0% = top one only (consistent results), closer to 100% means more randomness (more "creativity").
- vorticalbox 3y agoso a glorified Markov chain?
- TeMPOraL 3y agoYes, in the same sense a modern digital camera is a glorified photodiode. In both cases, light comes in, voltage comes out, and we can use it to count how much light came in.
- mirekrusin 3y agoWhy stop there, it's just ones and zeroes. It's "glorified markov chain" in the same sense that sqlite is just "glorified bubble sort".
- visarga 3y agoDon't you know that "attention is all you need"? Attention is non-markovian. It's all-to-all with some masking, not a chain.
- joshspankit 3y agoSince we’re here: Does a “resused” token count as a second token? For example: if you limited all inout/output to the same 100 words, could you stay within the token limit permanently?
- bhy 3y agoYes.
- lucubratory 3y agoBasically, yes! There are some technical quibbles (tokens don't line up to words, there's recent research on how to make LLMs stateful so they don't need to pass back the entire conversation for every token but can simply initialise to a remembered state), but this is basically correct as to what LLMs are mechanically doing. The really exciting stuff is how they're doing that! Research into that question is generally called interpretability, and it's probably what I'm most interested in at the moment.
- Dwolb 3y agoHow are they doing that?
- fooker 3y agoYou can get a Turing award by definitively answering this.
- lucubratory 3y agoWe don't now for sure, which is why it's so interesting to research! There are some tantalising hints though, for example the OthelloGPT results (here: https://thegradient.pub/othello/ https://thegradient.pub/othello/) which show that LLMs can form internal world models and then use them in inference to make decisions. Another interesting result is this one (https://www.nature.com/articles/s42003-022-03036-1 https://www.nature.com/articles/s42003-022-03036-1), which shows that neuronal activations in some parts of the human brain linearly map to neuronal activations in deep learning models when they are both doing the same task; this doesn't tell us anything about how either is doing what they are doing, but it does suggest that the human brain and LLMs may be doing a similar thing to achieve the same end result. We definitely need a lot more research before we can say anything definitive about this question, but there are a bunch of useful research directions that AI and neuroscience researchers are pursuing.
- idkyall 3y agoSome interesting follow-ups on OthelloGPT https://www.neelnanda.io/mechanistic-interpretability/othello https://www.neelnanda.io/mechanistic-interpretability/othell...
- ChatGTP 3y agoYou should get an award.
- golol 3y agoYes, at least decoder-only transformers afaik.
- deleted 3y ago[deleted]
- macrolime 3y agoThat's how autoregressive LLMs work. GPTs are autoregressive, but it's not the only way an LLM can work. You can also have other types of LLM, like diffusion based, other types should also be possible. At the moment, image models are mostly diffusion models while LLMs are mostly autoregressive. With a few other examples. Google has for example created both an autoregressive image model, Parti, and a diffusion model, Imagen. It's still early for non-autoregressive LLMs.
- maksimur 3y agoCould you or someone else kindly explain how it works for diffusion based LLMs, based on the style of GP? What happens for instance, if I give as input the prompt 'Describe Berlin' or maybe the more fitting 'Berlin city on a sunny day'?
- foota 3y agoI think you'd start out with gibberish and at each step the output would become more refined, so first you'd maybe see scrambled words emblematic of Berlin, which would then eventually synthesize into a sentence. I think you'd have a fixed length of text, with maybe some way for the model to mark some of it as truncated.
- 317070 3y agoMany diffusion based approaches have been tried for language. I like a diffusion style that goes a bit like this: * empty string * Berlin * Berlin city * Berlin is a city * Berlin is a city in Germany * Berlin is the capital of Germany * Berlin is the capital of Germany, located in the North East. * Berlin is the capital. It is located in the North East of Germany. * Berlin is the populous capital city of Germany. It is in the North East of Germany, by the river Spree. * Berlin is the capital city of Germany with 3.6 million inhabitants. It is located in the North East of Germany. It's centred on the river Spree. ... At every step, the diffusion slightly rephrases to add more information content and detail to that of the previous level, by inserting, replacing and deleting tokens. It is usually more costly to sample from these diffusion style models (which is not the case for diffusion for images, as unlike text, images tend to have a fixed size). But, one might imagine this approach to scale up to generate texts of arbitrary lengths. Just keep on adding more and more detail on your text, and you'll end up with a book or a coherent novel.
- imjonse 3y agoYes. "What is the most probable next word given the text so far" is the standard 'language modeling' task from classical NLP where it was done before using Markov chains and n-grams. RNNs then transformers and huge amounts of training data made them output what we see now, versus coherent but otherwise not very impressive text 10 years ago. The large contexts of current models made it possible to generate valid code where you need to remember to close a bracket opened 200 tokens before.
- IanCal 3y agoOne addition is that they don't return just one token, but the probabilities for each token. You could then greedily take the most probable single token, or select a bit more randomly (that's where temperature comes in) in a few different ways. You can do better still by exploring a few of the possible top options to select a high probability chain of tokens. That's because the most probable next token may not be the start of the most probable sequence of tokens. That's called beam search. Other than that, things like gpt can't go back and edit what they've outputted, so it's much more like listening to someone talk from the top of their head and writing down everything they say. They can say something wrong, followed by correcting themselves. The fact that the output becomes the input makes it much more obvious why "lets think through step by step" helps them reason, they can do simpler things then see the simpler steps they've already done to answer the more complex parts.
- PartiallyTyped 3y ago> That's called beam search. There are other decoding approaches, other than beam search: https://arxiv.org/abs/2202.00666 https://arxiv.org/abs/2202.00666 > Other than that, things like gpt can't go back and edit what they've outputted Actually, they can, they may emit a backspace token [1] [1] https://arxiv.org/abs/2306.05426 https://arxiv.org/abs/2306.05426, https://news.ycombinator.com/item?id=36425375 https://news.ycombinator.com/item?id=36425375
- james-revisoai 3y agoThe probabilities are fun (and ability to manipulate/mask/logit bias them), but small nit pick: Beam search isn't popular because it results in less human like text and takes more processing time, as a concept it sounds great in theory, but it's generally worked worse in practise compared to the next token with temperature, and results in text that is not human like, the gap is larger with larger models too, it doesn't get better: https://blog.fastforwardlabs.com/images/2019/05/Screen_Shot_2019_05_08_at_3_06_36_PM-1557342561886.png https://blog.fastforwardlabs.com/images/2019/05/Screen_Shot_... The way most GPT models are set up, they don't edit, but there was a lot of work (made redundant by OpenAI) on using gap filling tasks (masking) to iteratively improve or construct or otherwise edit and formulate logic using the encoder-decoder models of yesteryear - in many ways, I think it would have been more exciting if that approach powered the likes of 'think step-by-step' results.
- lynx23 3y agoYes.
- kolinko 3y agoWith LLAMA at least, not just past tokens, but also past states for each layer and each token. So LLama has information not just on the word "Berlin", but also on why it outputted "Berlin".
- YetAnotherNick 3y agoThat is called caching, which is used to save computation. But only tokens are enough to calculate next token and state could be recomputed.
- quickthrower2 3y agoYes. The actual output is all the tokens and a probability for each. You can pick the highest or take one randomly using those probabilities. Depends how creative or deterministic you want it to be.
- cubefox 3y agoExactly. A temperature of 0 means you always pick the highest probability token (i.e. the "max" function), while a temperature of 1 means you randomly pick a token according to their given probability. Values in between are also possible. However, it's important to note that the numbers which the models returns for the next tokens can only be interpreted as probabilities for foundation models. For fine-tuned models (instruction SL, RLHF) the numbers represent how good the model judges the next token to be. This also leads to a phenomenon called mode collapse.
- quickthrower2 3y agoVery interesting to know! (about FM vs FTM) Thanks.
- deleted 3y ago[deleted]
- woadwarrior01 3y agoAre you sure this is ChatGPT? If so, which version? I was under the impression that it's much better. Even the tiny 2.8B parameter LLM (fine tuned RedPajama-INCITE-Chat-3B) in my offline LLM app seems to do better (pardon the shameless plug!). > Human: Describe Berlin > Bot: Berlin is a city in Germany. It has been the capital of Germany since 1871 and it's also known as "the city that never sleeps". https://imgur.com/a/sBVHvxr https://imgur.com/a/sBVHvxr
- simonw 3y agoI don't think they were quoting an actual ChatGPT output there, just using an invented sentence to illustrate the concept of word-by-word generation.
- naveen99 3y agodon’t humans work the same way ?