5 ms·

> you could simply pluck out of your library the most common strings of word that seem to follow the words that were in your question! This is not sufficient t

> you could simply pluck out of your library the most common strings of word that seem to follow the words that were in your question! This is not sufficient to explain how LLMs are able to synthesize novel, coherent poems or song lyrics. What you're describing seems closer to a markov model. So far I have yet to see a good explanatin of _why_ transformer models seem to have this emergent behavior as you scale it up. It's easy to see by construction that the stacked attention layers will work well to predict the masked token in "Capital of France is [MASK]", or how you can throw an additional head on top for sentiment analysis (which I recall was implemented by literally adding a dummy token at the start into which information about the entire sentence gets embedded). But it's not obvious (to me at least) that this would somehow generalize into being able to do the things chatGPT does.

dtagames 4y ago
Ah, but in fact it does. Code and poetry seem different to people but tokens are tokens to the computer, and it knows not one from the other. The reason you get a poetry-style answer when you ask for one is that the mix of words and styles can be taken from different parts of the corpus. This is how you can have it write (bad) poetry about something for which only prose was scanned.
- nullish_signal 4y ago
  I think that the levels of Associativity between Linguistic Terms goes beyond a mere "mix of words and styles". When pulling on one part of the Prompt causes portions of the Output to shift or vanish, or change genre or word choice or composition or focal point or focal distance - - - then, that's the very opposite of "mix" -That's "Stable Diffusion" in text2img, or the "Reversal of Entropy" in Physics terms.. I'm still blown away that computers have become inage "Imaginers", decades after Humans first started doing 3D through Computers. It's really all quite unbelievable.
- maxbond 4y ago
  I'd propose that you come up with a prompt to test your hypothesis and try it out. I think seeing is believing here, I don't know how ChatGPT works but after experimenting with it I am sure it's not simply looking for runs of similar words to the prompt in it's training set. Here's an example to consider: https://news.ycombinator.com/item?id=33874110 https://news.ycombinator.com/item?id=33874110
throwaway1851 4y ago
I don't really see how it's different from, say, a large convolutional neural network learning progressively higher-order features as you progress through the layers of the network. At the lowest layers it's learning simple edge filters, which get combined into shapes, which get combined into filters that activate on faces, which get combined in ways that can be recognized as "family portrait", and so on. Of course, transformers have some unique advantages in terms of having very large context windows, being very parallelizable, etc. When ChatGPT or any generative model produces output from a prompt, it's sampling from the (frozen) statistical structure it has learned. It makes sense that as you increase model capacity and the volume of training data, you can capture statistical patterns that occur at a very high level of abstraction. So, instead of just predicting the next token based on token-to-token patterns, it can predict the next token based on something resembling concept-to-concept patterns. (This somewhat demystifies the poetry generation / style transfer stuff that it can do. If you ask for a breach of contract complaint written as a sonnet, it can sample from both the patterns it's learned from legal documents and the patterns it's learned from poetry.) What I wonder is how far this track of scaling up can take us. At the end of the day, interacting with ChatGPT is not that different from "interacting" with y=3x+7 by plugging in a value for x. It's just a much, much larger function.
- krackers 4y ago
  Thanks, this is a nice explanation! Also found a paper related to emergence in LLMs: https://arxiv.org/pdf/2206.07682.pdf https://arxiv.org/pdf/2206.07682.pdf

dtagames 4y ago

Ah, but in fact it does. Code and poetry seem different to people but tokens are tokens to the computer, and it knows not one from the other. The reason you get a poetry-style answer when you ask for one is that the mix of words and styles can be taken from different parts of the corpus. This is how you can have it write (bad) poetry about something for which only prose was scanned.

nullish_signal 4y ago
I think that the levels of Associativity between Linguistic Terms goes beyond a mere "mix of words and styles". When pulling on one part of the Prompt causes portions of the Output to shift or vanish, or change genre or word choice or composition or focal point or focal distance - - - then, that's the very opposite of "mix" -That's "Stable Diffusion" in text2img, or the "Reversal of Entropy" in Physics terms.. I'm still blown away that computers have become inage "Imaginers", decades after Humans first started doing 3D through Computers. It's really all quite unbelievable.
maxbond 4y ago
I'd propose that you come up with a prompt to test your hypothesis and try it out. I think seeing is believing here, I don't know how ChatGPT works but after experimenting with it I am sure it's not simply looking for runs of similar words to the prompt in it's training set. Here's an example to consider: https://news.ycombinator.com/item?id=33874110 https://news.ycombinator.com/item?id=33874110

throwaway1851 4y ago

I don't really see how it's different from, say, a large convolutional neural network learning progressively higher-order features as you progress through the layers of the network. At the lowest layers it's learning simple edge filters, which get combined into shapes, which get combined into filters that activate on faces, which get combined in ways that can be recognized as "family portrait", and so on. Of course, transformers have some unique advantages in terms of having very large context windows, being very parallelizable, etc. When ChatGPT or any generative model produces output from a prompt, it's sampling from the (frozen) statistical structure it has learned. It makes sense that as you increase model capacity and the volume of training data, you can capture statistical patterns that occur at a very high level of abstraction. So, instead of just predicting the next token based on token-to-token patterns, it can predict the next token based on something resembling concept-to-concept patterns. (This somewhat demystifies the poetry generation / style transfer stuff that it can do. If you ask for a breach of contract complaint written as a sonnet, it can sample from both the patterns it's learned from legal documents and the patterns it's learned from poetry.) What I wonder is how far this track of scaling up can take us. At the end of the day, interacting with ChatGPT is not that different from "interacting" with y=3x+7 by plugging in a value for x. It's just a much, much larger function.

krackers 4y ago
Thanks, this is a nice explanation! Also found a paper related to emergence in LLMs: https://arxiv.org/pdf/2206.07682.pdf https://arxiv.org/pdf/2206.07682.pdf