This is a very good guide, although if it’s truly aimed at normies it’s still far too formal/mathematical in certain parts (ChatGPT itself could probably help y

This is a very good guide, although if it’s truly aimed at normies it’s still far too formal/mathematical in certain parts (ChatGPT itself could probably help you rewrite some of those sections in plainer terms). The ‘token window’ section does a fantastic job of answering “but how does it know?”. The ‘lobes of probability’ section does a fantastic of answering “but why does it lie?”. The ‘whole universes of possible meanings’ bit does an okay job of answering “but how does it understand”, however I think that part could be made more explicit. What made it click for me was https://borretti.me/article/and-yet-it-understands https://borretti.me/article/and-yet-it-understands - specifically: “Every pair of token sequences can, in principle, be stored in a lookup table. You could, in principle, have a lookup table so vast any finite conversation with it would be indistinguishable from talking to a human, … But it wouldn’t fit in the entire universe. And there is no compression scheme … that would make it fit. But GPT-3 masses next to nothing at 800GiB. “How is it so small, and yet capable of so much? Because it is forgetting irrelevant details. There is another term for this: abstraction. It is forming concepts. There comes a point in the performance to model size curve where the simpler hypothesis has to be that the model really does understand what it is saying, and we have clearly passed it.” If I was trying to explain that to normies, I would try to hijack the popular “autocomplete on steroids” refrain. Currently it seems like normies know “autocomplete for words”, and think when you put it on steroids you get “autocomplete for paragraphs”. Explain to them that actually, what you get is “autocomplete for meanings”. (Please feel free to use these ideas in your post if you like them, don’t even think about crediting me, I just want to see the water level rise!)