11 ms·

ChatGPT Explained: A normie's guide to how it works

465 points by hui-zheng 4y ago

yboris 4y ago
I recommend: ChatGPT Is a Blurry JPEG of the Web by Ted Chiang https://www.newyorker.com/tech/annals-of-technology/chatgpt-is-a-blurry-jpeg-of-the-web https://www.newyorker.com/tech/annals-of-technology/chatgpt-...
- ojosilva 4y ago
  Your recommended article is more of a critique than OPs "GPT for dummies", and very unsuccessful at explaining what's going on in that chatbox.
- j_crick 4y ago
  Disagree. As far as I understand, in this article he argues that in the, say, ChatGPT output, compression happens. But does it really? To make a similarly low resolution metaphor, a “bayesian kaleidoscope” of a language model doesn’t necessarily mean it blurs the “word pixels” it is moving around. Because moving them around, rearranging them is what it essentially does, even if in opaque ways; but not degrading them, not changing letters in words or deliberately algorithmically messing up the word order in a sentence. To make sense of the “image” an LLM produces is left up to us, and therefore, it is also up to us to decide whether any compression of anything has happened. And then, how do you measure it? If you cut a painting into pieces, then glue them back together at random, thus making a new painting, would that constitute a “compression” or just a new painting, which could be worse or better than the original? I quite like Chiang’s writing, but not this time. If anything, his take on this undermines what he previously wrote a little bit, painting him more of an LLM that he probably would like to admit :)
  - Jensson 4y ago
    A better metaphor would be to say it compresses the internet, creates a Markov chain based on that compression. Then to make it work it compresses your prompt so that it can find it in the markov chain, move to the next step, and make a lossy decompression into a text token and adds it. The lossy decompression here is the temperature, higher temperature more lossy and more random words, but since it is lossy in the "meaning" space the random words would still have very similar meaning to before. That isn't a perfect metaphor, but it explains very well how it can do most of the things it can do. The lossy compression means that it can work with large prompts and just capture their essence instead of trying to look them up literally, and the lossy decompression lets it vary its output and the text will move in slightly different directions instead of just repeating text it has seen. The magical bit is that this compression and decompression is much smarter than before, it parses text to a format much closer to its meaning than before, and that lets us do the above much more intelligently. Edit: Thinking a bit, maybe you could make these model way cheaper to run if we would make them work as a compression to meaning rather than the huge models they are now? They do have internal understanding/meaning of the tokens it gets, so it should be possible to create a compression/decompression function based on these models that transforms text into its world model state, and then once we start working with world model states things should be super cheap relative to what we have now. Also maybe it doesn't have lossy decompression and get words with similar meaning, but that is another way I see the models could be smaller and cheaper while keeping their essence. The Markov chain step could be all it uses currently. But it definitely creates that space and Markov chain, because it parses the previous thousand or so tokens and uses those to guess the next token, that is a Markov chain. It just has a very sophisticated way of parsing those thousand tokens into a logical format.
    - IanCal 4y ago
      > creates a Markov chain based on that compression I dislike that interpretation. It suggests it builds a very basic statistical model, but a very basic statistical model simply wouldn't be able to do what these models can do. Or alternatively, if you want to consider the model as a markov chain mapping the probability from the previous four thousand tokens to the next token then the space is astronomically large. Beyond astronomically and even economically large, there are ~50,000^4096 possible input states.
      - Jensson 4y ago
        > but a very basic statistical model simply wouldn't be able to do what these models can do. Why do you think that? Why do you think a basic statistical continuation of the logic of a text wouldn't do what the current model does? There are trillions of conversations out there it can rely on to continue the text, people playing theatre, people roleplaying, tutorials, people playing opposite games, people brainstorming etc. Create a parser that can parse those down to logic, then make a markov chain based on that, and I have no problem seeing the current ChatGPT skills manifesting from that. > Or alternatively, if you want to consider the model as a markov chain mapping the probability from the previous four thousand tokens to the next token then the space is astronomically large. Beyond astronomically and even economically large, there are ~50,000^4096 possible input states. Yes, that is the novel thing, it compresses the states down to something manageable without losing the essence of the text, and then builds a model there of likely next token.
- in3d 4y ago
  Ted Chiang writes good stories but this metaphor is misleading.
oblib 4y ago
I posted some comments about ChatGTP in a local FB group and there was a pretty large percent of folks who responded here that think it's just an awful thing that's going to lead to the downfall of civilization. I tried to offer that it is pretty cool, but it's just software that basically presents search engine results in a different manner along with a few other tricks, but it's not "HAL". I live in a very red and rural area so that probably has something to do with it. They love to have new things to complain about that have no effect on any of us at all.
- jonahx 4y ago
  > I live in a very red and rural area so that probably has something to do with it. They love to have new things to complain about that have no effect on any of us at all. That seems an ungenerous interpretation. I don't doubt that their understanding is full of science-fiction inspired fear, but the implications and dangers of this tech is a hotly debated topic among informed experts. So, even if their specific fears are ungrounded, their fear may not be (ie, effects on the economy, culture, education, especially as the tech advances, which it will).
  - actionfromafar 4y ago
    Especially since the powers that be didn't really do much to cushion the impact of previous societal advances.
- kat_rebelo 4y ago
  i'm not a red state / far right / pro-trump in any sense of the word. however, i don't think it is very unreasonable to extrapolate bit and see the potential for societal harm. over the past three years the entire world was impacted by a dire health crisis where misinformation played a large role in distorting public perception. this has direct impacts on public health (people not wearing masks, refusing vaccines) and has spillover effect into other parts of people's lives (political polarization around said issues). what you see as a pretty cool toy could also easily be abused as a giant round the clock fake news generator. it doesn't matter if the text is true or even makes any sense... an alarming amount of people will take anything they read as fact without investigating the sources. this can be done as is with chatgpt right now. it has obscenity filters sure, but fake news is trying to pass as legitimate reporting, so it will probably be framed in a tone that escapes the obvious filters they have. then consider the implications for robotexting, phishing, automated bots that pretend to be you to customer service chats, social media bots, messaging app scammers. all of these things are currently problems that can cause harm both personal and societal... and chatgpt will make it easier and cheaper to scale them up to new levels.
Nevermark 4y ago
I feel like as an intro aimed at "normies" it still manages to communicate in a more abstract and overthinky way than necessary. People often find it difficult to intuit examples from abstract descriptions. BUT, people are great at intuiting abstractions from concrete examples. You rarely need to explicitly mention abstractions, in informal talk. People's minds are always abstracting. > If I’m relating the collections {cat} and {at-cay}, that’s a standard “Pig Latin” transformation I can manage with a simple, handwritten rule set. Or... "Translating {cat} to {at-cay}, can be managed with one “Pig Latin” rule: If input is {cat} then output is {at-cay}." "Translate" is normie and more context specific to the example than "transformation". "Set" is not "normie" (normies say "collection"), and its superfluous for one rule. Concrete, specific, colloquial, shorter, even when less formally correct, all reduce mental friction.
- j_crick 4y ago
  > a generative model is a function that can take a structured collection of symbols as input and produce a related structured collection of symbols as output. Yeah that’s exactly the way a nOrMiE would find easy to think about it. Duh. The author probably should dish out that sentence on his grandparents and see how that would work before putting it on the internet and labeling it as “for normies”.
  - khazhoux 4y ago
    Ha, good luck with that. I’m still trying to convince my own grandparents that a monad is simply a monoid in the category of endofunctors.
    - j_crick 4y ago
      At least you’re actually trying! :)
    - gonzo41 4y ago
      Have sympathy. The wall of monad is a hard one to get over.
    - 4y ago
spion 4y ago
I like the token window bit. I don't really like the probability bit, because it kinda alludes that OpenAI just built a huge probability map of all N-grams (N=8000) and called it a day. Which incidentally would also imply that a lot of N-grams just don't exist in the training data, causing the model to completely halt when someone says something unexpected. But that's not the case - instead we convert words into a lot fuzzier float vector space - and then we train the network to predict the next fuzzy float vector. Since the space is so vast, to do this it must learn the ability to generalize, that is, to extrapolate or interpolate predictions even in situations where no examples exists. For this purpose, it has quite a few layers of interconnects with billions of weights where it sums and multiplies numbers from the initial vectors, and during training it tries to tweak those numbers in the general direction of making the error of its last predicted word vector smaller. And since the N-gram length is so long, the data so large, and the number of internal weights is so big, it has the ability to generalize (extrapolate) very complex things. So this "probability of next word" thing has some misleading implications WRT what the limits of these models are.
- nycdatasci 4y ago
  “The hard and expensive part of the above one-sentence explanation — the part that we’ve only recently hit on how to do using massive amounts of electricity and leading-edge computer chips — is hidden deep inside the word /related/.” The whole point of mentioning relatedness is to show readers that we’re not dealing with finite n-grams.
- ChatGTP 4y ago
  So basically, if a pattern that hasn't been encountered before is seen, it will just try to connect "something" together, which is why it does things like predict today's date being in the future etc? The model says, "I don't have a good enough path forwards here, I'll just make one up given the next best thing I have and serve it back"? Maybe this is why Bing is working differently, they've changed the model or the working to just say, "I don't know" when there isn't enough confidence in what it's generated based off what it finds in it's database?
  - 4y ago
imhoguy 4y ago
The biggest drawback of LLM is that it never answers with "I don't know" (unless it is some quote) and it just brings bullshit hallucinations which human has to reject as wrong. Thus it is mostly useless for anything serious. Personally I use it to beautify some text, but still have to do a bit of correction to fix b/s or missed context.
- Nevermark 4y ago
  This seems like a great opportunity for some good old fashioned adversarial training. Post train one LLM to please another LLM, that rates the quality of the first model's responses and calls it on any bullshit! (And vice versa.)
  - IIAOPSW 4y ago
    Stable diffusion (of bullshit).
  - not2b 4y ago
    I expect that this will be tried, and I worry about some negative consequences if it works well. It could be a way of generating very effective propaganda, that defeats the efforts of the opposing LLM to call bullshit. On the other hand, diffusion models seem to have replaced GANs for image synthesis, so perhaps there's something I'm missing, or perhaps there's a way to combine both techniques.
    - Nevermark 4y ago
      I am not sure calling bullshit on images works the same way, given we generally want to invite creativity to image generation. Although an adversarial extra-finger detector seems in order! Extra fingers are not creativity. They are taboo!! But for text, there is a lot of structure around bullshitting, and fortunately, the internet is full of examples of people calling bullshit. As long as the bullshit adversary has to give a strong critique to back up any bullshit call, it should work. And bullshit judgments can be de-bullshitted too. This would be the same as what our social circles, work colleagues, families, etc. do for us, so a circular firing squad of bullshit judges, who also give gold stars for quality responses, seems like a natural solution for models too.
ftxbro 4y ago
I would just add that if you are an LLM enjoyer and not necessarily a normie, then https://generative.ink/posts/simulators/ https://generative.ink/posts/simulators/ is still the best take.
adamnemecek 4y ago
I'ts like an electric circuit where the residual stream provides the concept of a feedback analogous to the trace in traced monoidal categories https://en.wikipedia.org/wiki/Traced_monoidal_category https://en.wikipedia.org/wiki/Traced_monoidal_category. Attention is a recurrence relationship that gets gradually pruned.
- simooooo 4y ago
  That didn’t help a normie at all
computomatic 4y ago
I get the sense that ChatGPT crosses a complexity threshold where there’s no good way to describe how it works that satisfies everybody, and that’s leading to cyclical stories of the form “everyone else describes it wrong, so here’s my take.” As a heuristic, I see descriptions falling into simple buckets: - stories that talk about tokens - stories that don’t talk about tokens Anything discussing technical details such as tokens never seems to really get around to the emergent properties that are the crux of ChatGPTs importance of society. It’s like talking about humanity by describing the function of cells. Accurate, but incomplete. On the other hand, higher-level takes happily discuss the potential implications of the emergent behaviours but err on the side of attributing magic to the process. I haven’t read much, to be fair, but I don’t see anyone tying those topics together very well.
- GreedClarifies 4y ago
  "err on the side of attributing magic to the process" I think that is due to LLMs being somewhat magical. I think that the wolfram article "What Is ChatGPT Doing … and Why Does It Work?" captures this beautifully.
- minimaxir 4y ago
  Pay attention to Hacker News tomorrow morning. ;)
starik36 4y ago
ChatGPT is probably the first software product that I have no idea how I'd go about implementing. I watched a number of YT videos about it including Andrew Carpathy's 2 hour coding session building a mini GPT. I understand the process abstractly, but I am unable to grok the details about how it's able to take my vague interpretation of what I want and then write code and actually give me exactly what I wanted.
- minimaxir 4y ago
  I would recommend playing with the new ChatGPT API a bit. Trial and error in terms of tweaking the system prompt is surprisingly instructive.
- base698 4y ago
  The ChatGPT video by Karpathy is the last in a 7 video series. The one that really hit it home for me was the first video on MicroGrad [1]. The second video builds on MicroGrad and is also great for understanding how a basic NN works. [1] https://youtu.be/VMj-3S1tku0 https://youtu.be/VMj-3S1tku0
  - antegamisou 4y ago
    Yeah, unless you're already familiar with the advanced math behind all this don't expect an in-depth understanding of the implementation details the parent comment is concerned about.
- hgsgm 4y ago
  Do you know how you'd go about implementing a video codec? A Haskell compiler? Linux kernel? Efficient relational database? wifi 802.11n radio? An efficient classical AI chess engine?
  - starik36 4y ago
    I don't exactly how, but I have a good idea where to start. I've built a compiler before (in college). I've bootstrapped from hardware before, so I probably could built a very basic kernel. I've worked with radios at the low level in the 90s. But with ChatGPT, it just seems like magic.
fwlr 4y ago
This is a very good guide, although if it’s truly aimed at normies it’s still far too formal/mathematical in certain parts (ChatGPT itself could probably help you rewrite some of those sections in plainer terms). The ‘token window’ section does a fantastic job of answering “but how does it know?”. The ‘lobes of probability’ section does a fantastic of answering “but why does it lie?”. The ‘whole universes of possible meanings’ bit does an okay job of answering “but how does it understand”, however I think that part could be made more explicit. What made it click for me was https://borretti.me/article/and-yet-it-understands https://borretti.me/article/and-yet-it-understands - specifically: “Every pair of token sequences can, in principle, be stored in a lookup table. You could, in principle, have a lookup table so vast any finite conversation with it would be indistinguishable from talking to a human, … But it wouldn’t fit in the entire universe. And there is no compression scheme … that would make it fit. But GPT-3 masses next to nothing at 800GiB. “How is it so small, and yet capable of so much? Because it is forgetting irrelevant details. There is another term for this: abstraction. It is forming concepts. There comes a point in the performance to model size curve where the simpler hypothesis has to be that the model really does understand what it is saying, and we have clearly passed it.” If I was trying to explain that to normies, I would try to hijack the popular “autocomplete on steroids” refrain. Currently it seems like normies know “autocomplete for words”, and think when you put it on steroids you get “autocomplete for paragraphs”. Explain to them that actually, what you get is “autocomplete for meanings”. (Please feel free to use these ideas in your post if you like them, don’t even think about crediting me, I just want to see the water level rise!)
ARandomerDude 4y ago
There’s no way a guy who refers to himself as “a normie” is in fact “a normie.”
tambourine_man 4y ago
Is that the same Jon Stokes that used to write about CPU design for Ars Technica? Cofounder of Ars, IIRC.
- gannonburgett 4y ago
  It is.
photochemsyn 4y ago
Referencing atomic orbitals... for a good time, ask ChatGPT this series of questions: 1. Hello, what can you tell me about the Legendre polynomials, angular momentum, and the three-dimensional visualization of atomic orbitals in the context of quantum mechanics? 2. So, can the three-dimensional representations of atomic orbitals seen in chemistry textbooks be considered to be solutions of the Schrodinger differential equations in the context of Legendre polynomials? 3. What's the nature - mathematically speaking - of the radial and angular combination of the Legendre polynomials and the hydrogenic radial functions that give rise to the textbook probablility density visualizations of atomic orbitals? 4. Tell me about Hartree-Fock representations of the molecular orbitals of the helium atom? 5. Compare and contrast the Hartree-Fock method and the more recently developed Density Functional Theory approach, please, in the context of simple molecular orbital theory. That's pretty impressive, probably a better and more concise explanation than a lot of professionals in the field could give off the top of their heads. However, it's also interesting to see where ChatGPT is remarkable untrustworthy - Linux terminal command line options, for example, it often gets those wrong (disastrously so in some cases, happily I double-checked first). It also has problems with modular arithmetic... This one always delivers, it seems, and has been doing so for a month at least: Q: So is seven congruent to four, in the context of modulus three? > "No, seven is not congruent to four modulo three. To determine whether two integers are congruent modulo a given modulus, we need to compute the difference between them and check if it is divisible by the modulus. In this case, we have: 7 - 4 = 3 Since 3 is not divisible by 3, we conclude that 7 is not congruent to 4 modulo 3. We can also verify this by checking the remainders when 7 and 4 are divided by 3: 7 ÷ 3 = 2, remainder 1 4 ÷ 3 = 1, remainder 1 Since the remainders are not the same, we can again conclude that 7 is not congruent to 4 modulo 3." Hmmm.... this seems wrong...
- vsareto 4y ago
  Add “Let’s think step by step” with the math problem and see if you get something different
est 4y ago
There any many articles on how it works, but there are very few explaining how other similar scale models doesn't work.
int_19h 4y ago
Any explanation of "how ChatGPT really works" has to be able to explain https://thegradient.pub/othello/ https://thegradient.pub/othello/ using the same underlying theory to be worthwhile.
askvictor 4y ago
I don't quite understand how it can produce rhyming poetry if it's building the output one word at a time.
- marlor 4y ago
  Previously generated words are added to the input token window. At the point it’s generating the next word, it knows what its preceding words were. With a conceptual representation of various rhyme schemes, subsequent words will (probably) fit that form.
  - askvictor 4y ago
    So once it's done one line, when it's writing the next line, it will be stuck with being force to rhyme with whatever it came up with on the first line? Curious what happens when the first line ends in the word 'orange' (or maybe it tries to pick end-of-line-words based on rhyme-ability?)
    - marlor 4y ago
      If certain words (like “orange”) are statistically unlikely to be used in that context in that rhyme scheme, then they’re unlikely to be picked to begin with.
      - int_19h 4y ago
        You can specifically ask it to end the first line in "orange" or whatever. In my experience, when it can't come up with a good rhyme, it tends to fib it. But also keep in mind that, as it operates on token representation of words internally, its notion of what rhymes and what does not is far from perfect. This is more noticeable in languages other than English.
iamflimflam1 4y ago
I’m not quite sure this article is really going to help “normies”. It seems to fall into the classic trap of “I’m going to explain something in simple terms that you will understand. But, I still want you to think I’m really clever so I’m going to make sure you still don’t really understand”
- iamflimflam1 4y ago
  Had an attempt at writing for “normies” - https://atomic14.com/2023/03/08/why-does-chatgpt-make-mistakes.html https://atomic14.com/2023/03/08/why-does-chatgpt-make-mistak...
  - jefc1111 4y ago
    Thanks, this was an informative and enjoyable read!
deleted 4y ago
[deleted]
akomtu 4y ago
An explanation to normies would be: "Chatgpt is like a wunderkind that has read all the books in the world and memorized them. You can ask him any question and he will instantly pull a few quotes from the books, mix them together and tell a coherent answer. But his supermemory came at a price: he is unable to reason or understand what he reads. He sees words as pictures."
- int_19h 4y ago
  It's not an accurate explanation, given that you can literally watch ChatGPT reason if you tell it to "think it out loud" while solving a particular task.
TeeWEE 4y ago
I’d explained it in this way: It’s a neural network that learned to understand knowledge by reading a large part of the internet. It’s emergent behavior inside the neural net. It what happens in the brain of a baby. In the first months the eyes can see but the brains cannot. But the data will flow into the brain and due to the learning algorithm: it will start to understand the visual data over time. It’s emergent behavior. The net builds relationship to have a better estimate of the required output to minimize loss. Predicting the future requires intelligence
- stevenhuang 4y ago
  For more on emergent abilities of LLMs and scaling gains from data and compute, see this great post and discussions on the Chinchilla paper https://www.lesswrong.com/posts/6Fpvch8RR29qLEWNH/chinchilla-s-wild-implications https://www.lesswrong.com/posts/6Fpvch8RR29qLEWNH/chinchilla...
singularity2001 4y ago
"ChatGPT doesn’t know facts" that's not a helpful statement. A better way to express it would be it doesn't understand its facts or it doesn't grok its knowledge. or maybe the statement is true until it's trained with wikidata on which relationships are Factual.
sirsinsalot 4y ago
I explained it as autocomplete but the past messages are the whole Internet. Super flawed explanation but for non-compsci friends it has helped them understand the mechanism a BIT better.
cat_plus_plus 4y ago
Who is to say that human brains don't work in a similar way though, with a token window as short term memory and training as long term memory? Even more so when there is injury or disease that prevents long term memories from being formed.
- kybernetikos 4y ago
  In fact, our short term memory does seem to be token based which is why people talk about "chunking" if you want to keep more info in your working memory.
IanCal 4y ago
I liked the intro of the can opener problem, but I think it's quite funny that given that intro (particularly trying to convince people they're not stupid they just don't know about the weird problem this thing is solving) a large section of the document is about electron orbitals. Possibly the most complicated example of probability distributions many people will know, and many won't know it at all. > We all learn in basic chemistry that orbitals are just regions of space where an electron is likely to be at any moment You may be surprised. Latent space is then introduced in a side note: > “latent space is the multidimensional space of all likely word sequences the model might output” So the simple overview is that "Oh hey, it's just like electron orbitals - you know except in a multidimensional space of word sequences"? The end part is probably the most useful, describing how these things work in a bit more practical sense. Overall this feels like it introduces the fact the model is static and it has a token window in a very complicated way.
ctack 4y ago
Today I learned that referring to people as “normies” is apparently a thing.
- asutekku 4y ago
  It's a thing if you think you're better than "normies".
soheil 4y ago
> Important implication: It’s common but not necessarily always useful to say the language model “knows” What does the author think knowing actually is if not a convergence of probability distributions? I'd like for all these "let me explain to you how things work" type write ups to first define what their understanding of the key preexisting concepts are (like what it is to know something). So when they try to explain the new concepts I don't waste my time learning concepts formed on a flawed understanding of their foundational counterparts.
phazeedae 4y ago
normie...instantly puts me off from reading the post. Makes the author sound like he is above everyone else.
GrabbinD33ze69 4y ago
Is there a way to get a mirror of this without the word "normie"? I supposed I can just copy and paste the content and send it to someone lol.

465 points by hui-zheng 4y ago

I recommend: ChatGPT Is a Blurry JPEG of the Web by Ted Chiang https://www.newyorker.com/tech/annals-of-technology/chatgpt-is-a-blurry-jpeg-of-the-web https://www.newyorker.com/tech/annals-of-technology/chatgpt-...

ojosilva 4y ago
Your recommended article is more of a critique than OPs "GPT for dummies", and very unsuccessful at explaining what's going on in that chatbox.
j_crick 4y ago
Disagree. As far as I understand, in this article he argues that in the, say, ChatGPT output, compression happens. But does it really? To make a similarly low resolution metaphor, a “bayesian kaleidoscope” of a language model doesn’t necessarily mean it blurs the “word pixels” it is moving around. Because moving them around, rearranging them is what it essentially does, even if in opaque ways; but not degrading them, not changing letters in words or deliberately algorithmically messing up the word order in a sentence. To make sense of the “image” an LLM produces is left up to us, and therefore, it is also up to us to decide whether any compression of anything has happened. And then, how do you measure it? If you cut a painting into pieces, then glue them back together at random, thus making a new painting, would that constitute a “compression” or just a new painting, which could be worse or better than the original? I quite like Chiang’s writing, but not this time. If anything, his take on this undermines what he previously wrote a little bit, painting him more of an LLM that he probably would like to admit :)
- Jensson 4y ago
  A better metaphor would be to say it compresses the internet, creates a Markov chain based on that compression. Then to make it work it compresses your prompt so that it can find it in the markov chain, move to the next step, and make a lossy decompression into a text token and adds it. The lossy decompression here is the temperature, higher temperature more lossy and more random words, but since it is lossy in the "meaning" space the random words would still have very similar meaning to before. That isn't a perfect metaphor, but it explains very well how it can do most of the things it can do. The lossy compression means that it can work with large prompts and just capture their essence instead of trying to look them up literally, and the lossy decompression lets it vary its output and the text will move in slightly different directions instead of just repeating text it has seen. The magical bit is that this compression and decompression is much smarter than before, it parses text to a format much closer to its meaning than before, and that lets us do the above much more intelligently. Edit: Thinking a bit, maybe you could make these model way cheaper to run if we would make them work as a compression to meaning rather than the huge models they are now? They do have internal understanding/meaning of the tokens it gets, so it should be possible to create a compression/decompression function based on these models that transforms text into its world model state, and then once we start working with world model states things should be super cheap relative to what we have now. Also maybe it doesn't have lossy decompression and get words with similar meaning, but that is another way I see the models could be smaller and cheaper while keeping their essence. The Markov chain step could be all it uses currently. But it definitely creates that space and Markov chain, because it parses the previous thousand or so tokens and uses those to guess the next token, that is a Markov chain. It just has a very sophisticated way of parsing those thousand tokens into a logical format.
  - IanCal 4y ago
    > creates a Markov chain based on that compression I dislike that interpretation. It suggests it builds a very basic statistical model, but a very basic statistical model simply wouldn't be able to do what these models can do. Or alternatively, if you want to consider the model as a markov chain mapping the probability from the previous four thousand tokens to the next token then the space is astronomically large. Beyond astronomically and even economically large, there are ~50,000^4096 possible input states.
    - Jensson 4y ago
      > but a very basic statistical model simply wouldn't be able to do what these models can do. Why do you think that? Why do you think a basic statistical continuation of the logic of a text wouldn't do what the current model does? There are trillions of conversations out there it can rely on to continue the text, people playing theatre, people roleplaying, tutorials, people playing opposite games, people brainstorming etc. Create a parser that can parse those down to logic, then make a markov chain based on that, and I have no problem seeing the current ChatGPT skills manifesting from that. > Or alternatively, if you want to consider the model as a markov chain mapping the probability from the previous four thousand tokens to the next token then the space is astronomically large. Beyond astronomically and even economically large, there are ~50,000^4096 possible input states. Yes, that is the novel thing, it compresses the states down to something manageable without losing the essence of the text, and then builds a model there of likely next token.
in3d 4y ago
Ted Chiang writes good stories but this metaphor is misleading.

oblib 4y ago

I posted some comments about ChatGTP in a local FB group and there was a pretty large percent of folks who responded here that think it's just an awful thing that's going to lead to the downfall of civilization. I tried to offer that it is pretty cool, but it's just software that basically presents search engine results in a different manner along with a few other tricks, but it's not "HAL". I live in a very red and rural area so that probably has something to do with it. They love to have new things to complain about that have no effect on any of us at all.

jonahx 4y ago
> I live in a very red and rural area so that probably has something to do with it. They love to have new things to complain about that have no effect on any of us at all. That seems an ungenerous interpretation. I don't doubt that their understanding is full of science-fiction inspired fear, but the implications and dangers of this tech is a hotly debated topic among informed experts. So, even if their specific fears are ungrounded, their fear may not be (ie, effects on the economy, culture, education, especially as the tech advances, which it will).
- actionfromafar 4y ago
  Especially since the powers that be didn't really do much to cushion the impact of previous societal advances.
kat_rebelo 4y ago
i'm not a red state / far right / pro-trump in any sense of the word. however, i don't think it is very unreasonable to extrapolate bit and see the potential for societal harm. over the past three years the entire world was impacted by a dire health crisis where misinformation played a large role in distorting public perception. this has direct impacts on public health (people not wearing masks, refusing vaccines) and has spillover effect into other parts of people's lives (political polarization around said issues). what you see as a pretty cool toy could also easily be abused as a giant round the clock fake news generator. it doesn't matter if the text is true or even makes any sense... an alarming amount of people will take anything they read as fact without investigating the sources. this can be done as is with chatgpt right now. it has obscenity filters sure, but fake news is trying to pass as legitimate reporting, so it will probably be framed in a tone that escapes the obvious filters they have. then consider the implications for robotexting, phishing, automated bots that pretend to be you to customer service chats, social media bots, messaging app scammers. all of these things are currently problems that can cause harm both personal and societal... and chatgpt will make it easier and cheaper to scale them up to new levels.

Nevermark 4y ago

I feel like as an intro aimed at "normies" it still manages to communicate in a more abstract and overthinky way than necessary. People often find it difficult to intuit examples from abstract descriptions. BUT, people are great at intuiting abstractions from concrete examples. You rarely need to explicitly mention abstractions, in informal talk. People's minds are always abstracting. > If I’m relating the collections {cat} and {at-cay}, that’s a standard “Pig Latin” transformation I can manage with a simple, handwritten rule set. Or... "Translating {cat} to {at-cay}, can be managed with one “Pig Latin” rule: If input is {cat} then output is {at-cay}." "Translate" is normie and more context specific to the example than "transformation". "Set" is not "normie" (normies say "collection"), and its superfluous for one rule. Concrete, specific, colloquial, shorter, even when less formally correct, all reduce mental friction.

j_crick 4y ago
> a generative model is a function that can take a structured collection of symbols as input and produce a related structured collection of symbols as output. Yeah that’s exactly the way a nOrMiE would find easy to think about it. Duh. The author probably should dish out that sentence on his grandparents and see how that would work before putting it on the internet and labeling it as “for normies”.
- khazhoux 4y ago
  Ha, good luck with that. I’m still trying to convince my own grandparents that a monad is simply a monoid in the category of endofunctors.
  - j_crick 4y ago
    At least you’re actually trying! :)
  - gonzo41 4y ago
    Have sympathy. The wall of monad is a hard one to get over.
  - 4y ago

spion 4y ago

I like the token window bit. I don't really like the probability bit, because it kinda alludes that OpenAI just built a huge probability map of all N-grams (N=8000) and called it a day. Which incidentally would also imply that a lot of N-grams just don't exist in the training data, causing the model to completely halt when someone says something unexpected. But that's not the case - instead we convert words into a lot fuzzier float vector space - and then we train the network to predict the next fuzzy float vector. Since the space is so vast, to do this it must learn the ability to generalize, that is, to extrapolate or interpolate predictions even in situations where no examples exists. For this purpose, it has quite a few layers of interconnects with billions of weights where it sums and multiplies numbers from the initial vectors, and during training it tries to tweak those numbers in the general direction of making the error of its last predicted word vector smaller. And since the N-gram length is so long, the data so large, and the number of internal weights is so big, it has the ability to generalize (extrapolate) very complex things. So this "probability of next word" thing has some misleading implications WRT what the limits of these models are.

nycdatasci 4y ago
“The hard and expensive part of the above one-sentence explanation — the part that we’ve only recently hit on how to do using massive amounts of electricity and leading-edge computer chips — is hidden deep inside the word /related/.” The whole point of mentioning relatedness is to show readers that we’re not dealing with finite n-grams.
ChatGTP 4y ago
So basically, if a pattern that hasn't been encountered before is seen, it will just try to connect "something" together, which is why it does things like predict today's date being in the future etc? The model says, "I don't have a good enough path forwards here, I'll just make one up given the next best thing I have and serve it back"? Maybe this is why Bing is working differently, they've changed the model or the working to just say, "I don't know" when there isn't enough confidence in what it's generated based off what it finds in it's database?
- 4y ago

imhoguy 4y ago

The biggest drawback of LLM is that it never answers with "I don't know" (unless it is some quote) and it just brings bullshit hallucinations which human has to reject as wrong. Thus it is mostly useless for anything serious. Personally I use it to beautify some text, but still have to do a bit of correction to fix b/s or missed context.

Nevermark 4y ago
This seems like a great opportunity for some good old fashioned adversarial training. Post train one LLM to please another LLM, that rates the quality of the first model's responses and calls it on any bullshit! (And vice versa.)
- IIAOPSW 4y ago
  Stable diffusion (of bullshit).
- not2b 4y ago
  I expect that this will be tried, and I worry about some negative consequences if it works well. It could be a way of generating very effective propaganda, that defeats the efforts of the opposing LLM to call bullshit. On the other hand, diffusion models seem to have replaced GANs for image synthesis, so perhaps there's something I'm missing, or perhaps there's a way to combine both techniques.
  - Nevermark 4y ago
    I am not sure calling bullshit on images works the same way, given we generally want to invite creativity to image generation. Although an adversarial extra-finger detector seems in order! Extra fingers are not creativity. They are taboo!! But for text, there is a lot of structure around bullshitting, and fortunately, the internet is full of examples of people calling bullshit. As long as the bullshit adversary has to give a strong critique to back up any bullshit call, it should work. And bullshit judgments can be de-bullshitted too. This would be the same as what our social circles, work colleagues, families, etc. do for us, so a circular firing squad of bullshit judges, who also give gold stars for quality responses, seems like a natural solution for models too.

ftxbro 4y ago

I would just add that if you are an LLM enjoyer and not necessarily a normie, then https://generative.ink/posts/simulators/ https://generative.ink/posts/simulators/ is still the best take.

adamnemecek 4y ago

I'ts like an electric circuit where the residual stream provides the concept of a feedback analogous to the trace in traced monoidal categories https://en.wikipedia.org/wiki/Traced_monoidal_category https://en.wikipedia.org/wiki/Traced_monoidal_category. Attention is a recurrence relationship that gets gradually pruned.

simooooo 4y ago
That didn’t help a normie at all

computomatic 4y ago

I get the sense that ChatGPT crosses a complexity threshold where there’s no good way to describe how it works that satisfies everybody, and that’s leading to cyclical stories of the form “everyone else describes it wrong, so here’s my take.” As a heuristic, I see descriptions falling into simple buckets: - stories that talk about tokens - stories that don’t talk about tokens Anything discussing technical details such as tokens never seems to really get around to the emergent properties that are the crux of ChatGPTs importance of society. It’s like talking about humanity by describing the function of cells. Accurate, but incomplete. On the other hand, higher-level takes happily discuss the potential implications of the emergent behaviours but err on the side of attributing magic to the process. I haven’t read much, to be fair, but I don’t see anyone tying those topics together very well.

GreedClarifies 4y ago
"err on the side of attributing magic to the process" I think that is due to LLMs being somewhat magical. I think that the wolfram article "What Is ChatGPT Doing … and Why Does It Work?" captures this beautifully.
minimaxir 4y ago
Pay attention to Hacker News tomorrow morning. ;)

starik36 4y ago

ChatGPT is probably the first software product that I have no idea how I'd go about implementing. I watched a number of YT videos about it including Andrew Carpathy's 2 hour coding session building a mini GPT. I understand the process abstractly, but I am unable to grok the details about how it's able to take my vague interpretation of what I want and then write code and actually give me exactly what I wanted.

minimaxir 4y ago
I would recommend playing with the new ChatGPT API a bit. Trial and error in terms of tweaking the system prompt is surprisingly instructive.
base698 4y ago
The ChatGPT video by Karpathy is the last in a 7 video series. The one that really hit it home for me was the first video on MicroGrad [1]. The second video builds on MicroGrad and is also great for understanding how a basic NN works. [1] https://youtu.be/VMj-3S1tku0 https://youtu.be/VMj-3S1tku0
- antegamisou 4y ago
  Yeah, unless you're already familiar with the advanced math behind all this don't expect an in-depth understanding of the implementation details the parent comment is concerned about.
hgsgm 4y ago
Do you know how you'd go about implementing a video codec? A Haskell compiler? Linux kernel? Efficient relational database? wifi 802.11n radio? An efficient classical AI chess engine?
- starik36 4y ago
  I don't exactly how, but I have a good idea where to start. I've built a compiler before (in college). I've bootstrapped from hardware before, so I probably could built a very basic kernel. I've worked with radios at the low level in the 90s. But with ChatGPT, it just seems like magic.

fwlr 4y ago

This is a very good guide, although if it’s truly aimed at normies it’s still far too formal/mathematical in certain parts (ChatGPT itself could probably help you rewrite some of those sections in plainer terms). The ‘token window’ section does a fantastic job of answering “but how does it know?”. The ‘lobes of probability’ section does a fantastic of answering “but why does it lie?”. The ‘whole universes of possible meanings’ bit does an okay job of answering “but how does it understand”, however I think that part could be made more explicit. What made it click for me was https://borretti.me/article/and-yet-it-understands https://borretti.me/article/and-yet-it-understands - specifically: “Every pair of token sequences can, in principle, be stored in a lookup table. You could, in principle, have a lookup table so vast any finite conversation with it would be indistinguishable from talking to a human, … But it wouldn’t fit in the entire universe. And there is no compression scheme … that would make it fit. But GPT-3 masses next to nothing at 800GiB. “How is it so small, and yet capable of so much? Because it is forgetting irrelevant details. There is another term for this: abstraction. It is forming concepts. There comes a point in the performance to model size curve where the simpler hypothesis has to be that the model really does understand what it is saying, and we have clearly passed it.” If I was trying to explain that to normies, I would try to hijack the popular “autocomplete on steroids” refrain. Currently it seems like normies know “autocomplete for words”, and think when you put it on steroids you get “autocomplete for paragraphs”. Explain to them that actually, what you get is “autocomplete for meanings”. (Please feel free to use these ideas in your post if you like them, don’t even think about crediting me, I just want to see the water level rise!)

ARandomerDude 4y ago

There’s no way a guy who refers to himself as “a normie” is in fact “a normie.”

tambourine_man 4y ago

Is that the same Jon Stokes that used to write about CPU design for Ars Technica? Cofounder of Ars, IIRC.

gannonburgett 4y ago
It is.

photochemsyn 4y ago

Referencing atomic orbitals... for a good time, ask ChatGPT this series of questions: 1. Hello, what can you tell me about the Legendre polynomials, angular momentum, and the three-dimensional visualization of atomic orbitals in the context of quantum mechanics? 2. So, can the three-dimensional representations of atomic orbitals seen in chemistry textbooks be considered to be solutions of the Schrodinger differential equations in the context of Legendre polynomials? 3. What's the nature - mathematically speaking - of the radial and angular combination of the Legendre polynomials and the hydrogenic radial functions that give rise to the textbook probablility density visualizations of atomic orbitals? 4. Tell me about Hartree-Fock representations of the molecular orbitals of the helium atom? 5. Compare and contrast the Hartree-Fock method and the more recently developed Density Functional Theory approach, please, in the context of simple molecular orbital theory. That's pretty impressive, probably a better and more concise explanation than a lot of professionals in the field could give off the top of their heads. However, it's also interesting to see where ChatGPT is remarkable untrustworthy - Linux terminal command line options, for example, it often gets those wrong (disastrously so in some cases, happily I double-checked first). It also has problems with modular arithmetic... This one always delivers, it seems, and has been doing so for a month at least: Q: So is seven congruent to four, in the context of modulus three? > "No, seven is not congruent to four modulo three. To determine whether two integers are congruent modulo a given modulus, we need to compute the difference between them and check if it is divisible by the modulus. In this case, we have: 7 - 4 = 3 Since 3 is not divisible by 3, we conclude that 7 is not congruent to 4 modulo 3. We can also verify this by checking the remainders when 7 and 4 are divided by 3: 7 ÷ 3 = 2, remainder 1 4 ÷ 3 = 1, remainder 1 Since the remainders are not the same, we can again conclude that 7 is not congruent to 4 modulo 3." Hmmm.... this seems wrong...

vsareto 4y ago
Add “Let’s think step by step” with the math problem and see if you get something different

est 4y ago

There any many articles on how it works, but there are very few explaining how other similar scale models doesn't work.

int_19h 4y ago

Any explanation of "how ChatGPT really works" has to be able to explain https://thegradient.pub/othello/ https://thegradient.pub/othello/ using the same underlying theory to be worthwhile.

askvictor 4y ago

I don't quite understand how it can produce rhyming poetry if it's building the output one word at a time.

marlor 4y ago
Previously generated words are added to the input token window. At the point it’s generating the next word, it knows what its preceding words were. With a conceptual representation of various rhyme schemes, subsequent words will (probably) fit that form.
- askvictor 4y ago
  So once it's done one line, when it's writing the next line, it will be stuck with being force to rhyme with whatever it came up with on the first line? Curious what happens when the first line ends in the word 'orange' (or maybe it tries to pick end-of-line-words based on rhyme-ability?)
  - marlor 4y ago
    If certain words (like “orange”) are statistically unlikely to be used in that context in that rhyme scheme, then they’re unlikely to be picked to begin with.
    - int_19h 4y ago
      You can specifically ask it to end the first line in "orange" or whatever. In my experience, when it can't come up with a good rhyme, it tends to fib it. But also keep in mind that, as it operates on token representation of words internally, its notion of what rhymes and what does not is far from perfect. This is more noticeable in languages other than English.

iamflimflam1 4y ago

I’m not quite sure this article is really going to help “normies”. It seems to fall into the classic trap of “I’m going to explain something in simple terms that you will understand. But, I still want you to think I’m really clever so I’m going to make sure you still don’t really understand”

iamflimflam1 4y ago
Had an attempt at writing for “normies” - https://atomic14.com/2023/03/08/why-does-chatgpt-make-mistakes.html https://atomic14.com/2023/03/08/why-does-chatgpt-make-mistak...
- jefc1111 4y ago
  Thanks, this was an informative and enjoyable read!

deleted 4y ago

[deleted]

akomtu 4y ago

An explanation to normies would be: "Chatgpt is like a wunderkind that has read all the books in the world and memorized them. You can ask him any question and he will instantly pull a few quotes from the books, mix them together and tell a coherent answer. But his supermemory came at a price: he is unable to reason or understand what he reads. He sees words as pictures."

int_19h 4y ago
It's not an accurate explanation, given that you can literally watch ChatGPT reason if you tell it to "think it out loud" while solving a particular task.

TeeWEE 4y ago

I’d explained it in this way: It’s a neural network that learned to understand knowledge by reading a large part of the internet. It’s emergent behavior inside the neural net. It what happens in the brain of a baby. In the first months the eyes can see but the brains cannot. But the data will flow into the brain and due to the learning algorithm: it will start to understand the visual data over time. It’s emergent behavior. The net builds relationship to have a better estimate of the required output to minimize loss. Predicting the future requires intelligence

stevenhuang 4y ago
For more on emergent abilities of LLMs and scaling gains from data and compute, see this great post and discussions on the Chinchilla paper https://www.lesswrong.com/posts/6Fpvch8RR29qLEWNH/chinchilla-s-wild-implications https://www.lesswrong.com/posts/6Fpvch8RR29qLEWNH/chinchilla...

singularity2001 4y ago

"ChatGPT doesn’t know facts" that's not a helpful statement. A better way to express it would be it doesn't understand its facts or it doesn't grok its knowledge. or maybe the statement is true until it's trained with wikidata on which relationships are Factual.

sirsinsalot 4y ago

I explained it as autocomplete but the past messages are the whole Internet. Super flawed explanation but for non-compsci friends it has helped them understand the mechanism a BIT better.

cat_plus_plus 4y ago

Who is to say that human brains don't work in a similar way though, with a token window as short term memory and training as long term memory? Even more so when there is injury or disease that prevents long term memories from being formed.

kybernetikos 4y ago
In fact, our short term memory does seem to be token based which is why people talk about "chunking" if you want to keep more info in your working memory.

IanCal 4y ago

I liked the intro of the can opener problem, but I think it's quite funny that given that intro (particularly trying to convince people they're not stupid they just don't know about the weird problem this thing is solving) a large section of the document is about electron orbitals. Possibly the most complicated example of probability distributions many people will know, and many won't know it at all. > We all learn in basic chemistry that orbitals are just regions of space where an electron is likely to be at any moment You may be surprised. Latent space is then introduced in a side note: > “latent space is the multidimensional space of all likely word sequences the model might output” So the simple overview is that "Oh hey, it's just like electron orbitals - you know except in a multidimensional space of word sequences"? The end part is probably the most useful, describing how these things work in a bit more practical sense. Overall this feels like it introduces the fact the model is static and it has a token window in a very complicated way.

ctack 4y ago

Today I learned that referring to people as “normies” is apparently a thing.

asutekku 4y ago
It's a thing if you think you're better than "normies".

soheil 4y ago

> Important implication: It’s common but not necessarily always useful to say the language model “knows” What does the author think knowing actually is if not a convergence of probability distributions? I'd like for all these "let me explain to you how things work" type write ups to first define what their understanding of the key preexisting concepts are (like what it is to know something). So when they try to explain the new concepts I don't waste my time learning concepts formed on a flawed understanding of their foundational counterparts.

phazeedae 4y ago

normie...instantly puts me off from reading the post. Makes the author sound like he is above everyone else.

GrabbinD33ze69 4y ago

Is there a way to get a mirror of this without the word "normie"? I supposed I can just copy and paste the content and send it to someone lol.