7 ms·

The Illustrated Transformer

500 points by auraham 10mo ago

profsummergig 10mo ago
Haven't watched it yet... ...but, if you have favorite resources on understanding Q & K, please drop them in comments below... (I've watched the Grant Sanderson/3blue1brown videos [including his excellent talk at TNG Big Tech Day '24], but Q & K still escape me). Thank you in advance.
- red2awn 10mo ago
  Implement transformers yourself (ie in Numpy). You'll never truly understand it by just watching videos.
  - D-Machine 10mo ago
    Seconding this, the terms "Query" and "Value" are largely arbitrary and meaningless in practice, look at how to implement this in PyTorch and you'll see these are just weight matrices that implement a projection of sorts, and self-attention is always just self_attention(x, x, x) or self_attention(x, x, y) in some cases, where x and y are are outputs from previous layers. Plus with different forms of attention, e.g. merged attention, and the research into why / how attention mechanisms might actually be working, the whole "they are motivated by key-value stores" thing starts to look really bogus. Really it is that the attention layer allows for modeling correlations and/or multiplicative interactions among a dimension-reduced representation.
    - profsummergig 10mo ago
      Do you think the dimension reduction is necessary? Or is it just practical (due to current hardware scarcity)?
      - D-Machine 10mo ago
        Definitely mostly just a practical thing IMO, especially with modern attention variants (sparse attention, FlashAttention, linear attention, merged attention etc). Not sure it is even hardware scarcity per se / solely, it would just be really expensive in terms of both memory and FLOPs (and not clearly increase model capacity) to use larger matrices. Also for the specific part where you, in code for encoder-decoder transformers, call the a(x, x, y) function instead of the usual a(x, x, x) attention call (what Alammar calls "encoder-decoder attention" in his diagram just before the "The Decoder Side"), you have different matrix sizes, so dimension reduction is needed to make the matrix multiplications work out nicely too. But in general it is just a compute thing IMO.
    - tayo42 10mo ago
      >the terms "Query" and "Value" are largely arbitrary and meaningless in practice This is the most confusing thing about it imo. Those words all mean something but they're just more matrix multiplications. Nothing was being searched for.
      - D-Machine 10mo ago
        Better resources will note the terms are just historical and not really relevant anymore, and just remain a naming convention for self-attention formulas. IMO it is harmful to learning and good pedagogy to say they are anything more than this, especially as we better understand the real thing they are doing is approximating feature-feature correlations / similarity matrices, or perhaps even more generally, just allow for multiplicative interactions (https://openreview.net/forum?id=rylnK6VtDH https://openreview.net/forum?id=rylnK6VtDH).
  - krat0sprakhar 10mo ago
    Do you have a tutorial that I can follow?
    - roadside_picnic 10mo ago
      The most valuable tutorial will be translating from the paper itself. The more hand holding you have in the process, the less you'll be learning conceptually. The pure manipulation of matrices is rather boring and uninformative without some context. I also think the implementation is more helpful for understanding the engineering work to run these models that getting a deeper mathematical understanding of what the model is doing.
    - jwitthuhn 10mo ago
      If you have 20 hours to spare I highly recommend this youtube playlist from Andrej Karpathy https://www.youtube.com/watch?v=VMj-3S1tku0&list=PLAqhIrjkxbuWI23v9cThsA9GvCAUhRvKZ https://www.youtube.com/watch?v=VMj-3S1tku0&list=PLAqhIrjkxb... It starts with the fundamentals of how backpropagation works then advances to building a few simple models and ends with building a GPT-2 clone. It won't taech you everything about AI models but it gives you a solid foundation for branching out.
  - roadside_picnic 10mo ago
    I personally don't think implementation is as enlightening as far as really understanding what the model is doing as this statement implies. I had done that many times, but it wasn't until reading about the relationship to kernel methods that it really clicked for me what is really happening under the hood. Don't get me wrong, implementing attention is still great (and necessary), but even with something as simple as linear regression, implementing it doesn't really give you the entire conceptual model. I do think implementation helps to understand the engineering of these models, but it still requires reflection and study to start to understand conceptually why they are working and what they're really doing (I would, of course, argue I'm still learning about linear models in that regard!)
- leopd 10mo ago
  I think this video does a pretty good job explaining it, starting about 10:30 minutes in: https://www.youtube.com/watch?v=S27pHKBEp30 https://www.youtube.com/watch?v=S27pHKBEp30
  - oofbey 10mo ago
    As the first comment says "This aged like fine wine". Six years old, but the fundamentals haven't changed.
  - andoando 10mo ago
    This wasn't any better than other explanation I've seen.
- throw310822 10mo ago
  Have you tried asking e.g. Claude to explain it to you? None of the usual resources worked for me, until I had a discussion with Claude where I could ask questions about everything that I didn't get.
  - sakesun 10mo ago
    Perhaps we have already reached ASI. :)
    - throw310822 10mo ago
      In some respects, yes. There is no single human being with a general knowledge as vast as that of a SOTA LLM, or able to speak as many languages. Claude knows about transformers more than enough to explain them to a layperson, elucidating specific points and resolving doubts. As someone who learns more easily by prodding other people's knowledge rather than from static explanations, I find LLMs extremely useful.
- bobbyschmidd 10mo ago
  tldr: recursively aggregating packing/unpacking 'if else if (functions)/statements' as keyword arguments that (call)/take them themselves as arguments, with their own position shifting according to the number "(weights)" of else if (functions)/statements needed to get all the other arguments into (one of) THE adequate orders. the order changes based on the language, input prompt and context. if I understand it all correctly. implemented it in html a while ago and might do it in htmx sometime soon. transformers are just slutty dictionaries that Papa Roach and kage bunshin no jutsu right away again and again, spawning clones and variations based on requirements, which is why they tend to repeat themselves rather quickly and often. it's got almost nothing to do with languages themselves and requirements and weights amount to playbooks and DEFCON levels
- roadside_picnic 10mo ago
  It's just a re-invention of kernel smoothing. Cosma Shalizi has an excellent write up on this [0]. Once you recognize this it's a wonderful re-framing of what a transformer is doing under the hood: you're effectively learning a bunch of sophisticated kernels (though the FF part) and then applying kernel smoothing in different ways through the attention layers. It makes you realize that Transformers are philosophically much closer to things like Gaussian Processes (which are also just a bunch of kernel manipulation). 0. http://bactra.org/notebooks/nn-attention-and-transformers.html http://bactra.org/notebooks/nn-attention-and-transformers.ht...
- machinationu 10mo ago
  Q, K and V are a way of filtering the relevant aspects for the task at hand from the token embeddings. "he was red" - maybe color, maybe angry, the "red" token embedding carries both, but only one aspect is relevant for some particular prompt. https://ngrok.com/blog/prompt-caching/ https://ngrok.com/blog/prompt-caching/
- oedemis 10mo ago
  there is also very good explanation from Luis Serrano, https://youtu.be/fkO9T027an0 https://youtu.be/fkO9T027an0
laser9 10mo ago
Here's the comment from the author himself (jayalammar) talking about other good resources on learning Transformers: https://news.ycombinator.com/item?id=35990118 https://news.ycombinator.com/item?id=35990118
boltzmann_ 10mo ago
Kudos also to Transformer Explainer team for putting some amazing visualizations https://poloclub.github.io/transformer-explainer/ https://poloclub.github.io/transformer-explainer/ It really clicked to me after reading this two and watching 3blue1brown videos
- gzer0 10mo ago
  This is hands down one of the best visualizations I have ever come across.
gustavoaca1997 10mo ago
I have this book. Really a life savior to help me catching up a few months ago when my team decided to use LLMs in our systems.
- qoez 10mo ago
  Don't really see why you'd need to understand how the transformer works to do LLMs at work. LLMs is just a synthetic human performing reasoning with some failure modes that in-depth knowledge of the transformer interals won't help you predict what they are (just have to use experience with the output to get a sense, or other peoples experiments).
  - roadside_picnic 10mo ago
    In my experience this is a substantial difference in the ability to really get performance in LLM related engineering work from people who really understand how LLMs work vs people who think it's a magic box. If your mental model of an LLM is: > a synthetic human performing reasoning You are severely overestimating the capabilities of these models and not realizing potential areas of failure (even if your prompt works for now in the happy case). Understanding how transformers work absolutely can help debug problems (or avoid them in the first place). People without a deep understanding of LLMs also tend to get fooled by them more frequently. When you have internalized the fact that LLMs are literally optimistized to trick you, you tend to be much more skeptical of the initial results (which results in better eval suites etc). Then there's people who actually do AI engineering. If you're working with local/open weights models or on the inference end of things you can't just play around with an API, you have a lot more control and observability into the model and should be making use of it. I still hold that the best test of an AI Engineer, at any level of the "AI" stack, is how well they understand speculative decoding. It involves understanding quite a bit about how LLMs work and can still be implemented on a cheap laptop.
    - amelius 10mo ago
      But that AI engineer who is implementing speculative decoding is still just doing basic plumbing that has little to do with the actual reasoning. Yes, he/she might make the process faster, but they will know just as little about why/how the reasoning works as when they implemented a naive, slow version of the inference.
Koshkin 10mo ago
(Going on a tangent.) The number of transformer explanations/tutorials is becoming overwhelming. Reminds me of monads (or maybe calculus). Someone feels a spark of enlightenment at some point (while, often, in fact, remaining deeply confused), and an urge to share their newly acquired (mis)understanding with a wide audience.
- nospice 10mo ago
  So? There's no rule that the internet is limited to a single explanation. Find the one that clicks for you, ignore the rest. Whenever I'm trying to learn about concepts in mathematics, computer science, physics, or electronics, I often find that the first or the "canonical" explanation is hard for me to parse. I'm thankful for having options 2 through 10.
- kadushka 10mo ago
  Maybe so, but this particular blog post was the first and is still the best explanation of how transformers work.
ActorNightly 10mo ago
People need to get away from this idea of Key/Query/Value as being special. Whereas a standard deep layer in a network is matrix * input, where each row of the matrix is the weights of the particular neuron in the next layer, a transformer is basically input* MatrixA, input*MatrixB, input*MatrixC (where vector*matrix is a matrix), then the output is C*MatrixA*MatrixB*MatrixC. Just simply more dimensions in a layer. And consequently, you can represent the entire transformer architecture with a set of deep layers as you unroll the matricies, with a lot of zeros for the multiplication pieces that are not needed. This is a fairly complex blog but it shows that its just all matrix multiplication all the way down. https://pytorch.org/blog/inside-the-matrix/ https://pytorch.org/blog/inside-the-matrix/.
- throw310822 10mo ago
  I might be completely off road, but I can't help thinking of convolutions as my mental model for the K Q V mechanism. Attention has the same property of a convolution kernel of being trained independently of position; it learns how to translate a large, rolling portion of an input to a new "digested" value; and you can train multiple ones in parallel so that they learn to focus on different aspects of the input ("kernels" in the case of convolution, "heads" in the case of attention).
  - krackers 10mo ago
    I think there are two key differences though: 1) Attention doesn't doesn't use fixed distance-dependent weight for the aggregation but instead the weight becomes "semantically-dependent", based on association between q/k. 2) A single convolution step is a local operation (only pulling from nearby pixels), whereas attention is a "global" operation, pulling from the hidden states of all previous tokens. (Maybe sliding window attention schemes muddy this distinction, but in general the degree of connectivity seems far higher). There might be some unifying way to look at things though, maybe GNNs. I found this talk [1] and at 4:17 it shows how convolution and attention would be modeled in a GNN formalism [1] https://www.youtube.com/watch?v=J1YCdVogd14 https://www.youtube.com/watch?v=J1YCdVogd14
    - 10mo ago
deleted 10mo ago
[deleted]
zkmon 10mo ago
I think the internal of transformers would become less relevant like internal of compilers, as programmers would only care about how to "use" them instead of how to develop them.
- rvz 10mo ago
  Their internals are just as relevant (now even more relevant) as any other technology as they always need to be improved to the SOTA (state of the art) meaning that someone has to understand their internals. It also means more jobs for the people who understand them at a deeper level to advance the SOTA of specific widely used technologies such as operating systems, compilers, neural network architectures and hardware such as GPUs or TPU chips. Someone has to maintain and improve them.
- esafak 10mo ago
  Practitioners already do not need to know about it to run let alone use LLMs. I bet most don't even know the fundamentals of machine learning. Hands up if you know bias from variance...
- crystal_revenge 10mo ago
  Have you written a compiler? I ask because for me writing a compiler was absolutely an inflection point in my journey as a programmer. Being able to look at code and reason about it all the way down to bytecode/IL/asm etc absolutely improved my skill as a programmer and ability to reason about software. For me this was the first time I felt like a real programmer.
  - zkmon 10mo ago
    Writing a compiler is not a requirement or good use of time for a programmer. Same as why driving a car should not require you to build the car engine. Driver should stick to their role and learn how to drive properly.
    - crystal_revenge 10mo ago
      I'm guessing the answer then is "no". That's a ridiculous metaphor as well because building a compiler is a massive software engineering project that covers a huge range of essential skills. That metaphor would work for building a computer, but not a compiler. Clearly it shouldn't be a requirement, but it is an excellent use of a programmer's time. I can think of no software project over my career that has improved my skills more than writing a compiler.
libraryofbabel 10mo ago
I read this article back when I was learning the basics of transformers; the visualizations were really helpful. Although in retrospect knowing how a transformer works wasn't very useful at all in my day job applying LLMs, except as a sort of deep background for reassurance that I had some idea of how the big black box producing the tokens was put together, and to give me the mathematical basis for things like context size limitations etc. I would strongly caution anyone who thinks that they will be able to understand or explain LLM behavior better by studying the architecture closely. That is a trap. Big SotA models these days exhibit so much nontrivial emergent phenomena (in part due to the massive application of reinforcement learning techniques) that give them capabilities very few people expected to ever see when this architecture first arrived. Most of us confidently claimed even back in 2023 that, based on LLM architecture and training algorithms, LLMs would never be able to perform well on novel coding or mathematics tasks. We were wrong. That points towards some caution and humility about using network architecture alone to reason about how LLMs work and what they can do. You'd really need to be able to poke at the weights inside a big SotA model to even begin to answer those kinds of questions, but unfortunately that's only really possible if you're a "mechanistic interpretability" researcher at one of the major labs. Regardless, this is a nice article, and this stuff is worth learning because it's interesting for its own sake! Right now I'm actually spending some vacation time implementing a transformer in PyTorch just to refresh my memory of it all. It's a lot of fun! If anyone else wants to get started with that I would highly recommend Sebastian Raschka's book and youtube videos as way into the subject: https://github.com/rasbt/LLMs-from-scratch https://github.com/rasbt/LLMs-from-scratch . Has anyone read TFA author Jay Alammar's book (published Oct 2024) and would they recommend it for a more up-to-date picture?
- nrhrjrjrjtntbt 10mo ago
  It is almost like understanding wood at a molecular level and being a carpenter. It also may help the carpentery, but you cam be a great one without it. And a bad one with the knowledge.
- ozgung 10mo ago
  I think the biggest problem is that most tutorials use words to illustrate how the attention mechanism works. In reality, there are no word-associated tokens inside a Transformer. Tokens != word parts. An LLM does not perform language processing inside the Transformer blocks, and a Vision Transformer does not perform image processing. Words and pixels are only relevant at the input. I think this misunderstanding was a root cause of underestimating their capabilities.
some_guy_nobel 10mo ago
Great article, must be the inspiration for the recent Illustrated Evo 2: https://research.nvidia.com/labs/dbr/blog/illustrated-evo2/ https://research.nvidia.com/labs/dbr/blog/illustrated-evo2/
prashant418 10mo ago
This guide is such a beast, Try pairing this guide with say claude code and ask it to generate sample mini pytorch pesudo-code and you can spend hours just learning/re-learning and mentally visualize a lot of these concepts. I am a big fan
edge17 10mo ago
Maybe I'm out of touch, but have transformers replaced all traditional deep learning architectures? (U-nets, etc)?
- D-Machine 10mo ago
  No, not at all. There is a transformer obsession that is quite possibly not supported by the actual facts (CNNs can still do just as well: https://arxiv.org/abs/2310.16764 https://arxiv.org/abs/2310.16764), and CNNs definitely remain preferable for smaller and more specialized tasks (e.g. computer vision on medical data). If you also get into more robust and/or specialized tasks (e.g. rotation invariant computer vision models, graph neural networks, models working on point-cloud data, etc) then transformers are also not obviously the right choice at all (or even usable in the first place). So plenty of other useful architectures out there.
  - edge17 10mo ago
    Is there something I can read to get a better sense of what types of models are most suitable for which problems? All I hear about are transformers nowadays, but what are the types of problems for which transformers are the right architecture choice?
    - D-Machine 10mo ago
      Just do some basic searches on e.g. Google Scholar for your task (e.g. "medical image segmentation", "point cloud segmentation", "graph neural networks", "timeseries classification", "forecasting") or task modification (e.g. "'rotation invariant' architecture") or whatever, sort by year, make sure to click on papers that have a large number of citations, and start reading. You will start to get a feel for domains or specific areas where transformers are and are not clearly the best models. Or just ask e.g. ChatGPT Thinking with search enabled about these kinds of things (and then verify the answer by going to the actual papers). Also check HuggingFace and other model hubs and filter by task to see if any of these models are available in an easy-to-use format. But most research models will only be available on GitHub somewhere, and in general you are just deciding between a vision transformer and the latest convolutional model (usually a ConvNext vX for some X). In practice, if you need to work with the kind of data that is found online, and don't have a highly specialized type of data or problem, then you do, today, almost always just want some pre-trained transformer. But if you actually have to (pre)train a model from scratch on specialized data, in many cases you will not have enough data or resources to get the most out of a transformer, and often some kind of older / simpler convolutional model is going to give better performance at less cost. Sometimes in these cases you don't even want a deep-learner at all, and just classic ML or algorithms are far superior. A good example would be timeseries forecasting, where embarrassingly simple linear models blow overly-complicated and hugely expensive transformer models right out of the water (https://arxiv.org/abs/2205.13504 https://arxiv.org/abs/2205.13504). Oh, right, and unless TabPFNv2 (https://www.nature.com/articles/s41586-024-08328-6 https://www.nature.com/articles/s41586-024-08328-6) makes sense for your use-case, you are still better off using boosted decision trees (e.g. XGBoost, LightGBM, or CatBoost) for tabular data.
Simplita 10mo ago
Visual explanations like this make it clearer why models struggle once context balloons. In practice, breaking problems into explicit stages helped us more than just increasing context length.
deleted 10mo ago
[deleted]
bearsortree 10mo ago
i found this much more intuitive to follow, https://poloclub.github.io/transformer-explainer/ https://poloclub.github.io/transformer-explainer/

500 points by auraham 10mo ago

Haven't watched it yet... ...but, if you have favorite resources on understanding Q & K, please drop them in comments below... (I've watched the Grant Sanderson/3blue1brown videos [including his excellent talk at TNG Big Tech Day '24], but Q & K still escape me). Thank you in advance.

red2awn 10mo ago
Implement transformers yourself (ie in Numpy). You'll never truly understand it by just watching videos.
- D-Machine 10mo ago
  Seconding this, the terms "Query" and "Value" are largely arbitrary and meaningless in practice, look at how to implement this in PyTorch and you'll see these are just weight matrices that implement a projection of sorts, and self-attention is always just self_attention(x, x, x) or self_attention(x, x, y) in some cases, where x and y are are outputs from previous layers. Plus with different forms of attention, e.g. merged attention, and the research into why / how attention mechanisms might actually be working, the whole "they are motivated by key-value stores" thing starts to look really bogus. Really it is that the attention layer allows for modeling correlations and/or multiplicative interactions among a dimension-reduced representation.
  - profsummergig 10mo ago
    Do you think the dimension reduction is necessary? Or is it just practical (due to current hardware scarcity)?
    - D-Machine 10mo ago
      Definitely mostly just a practical thing IMO, especially with modern attention variants (sparse attention, FlashAttention, linear attention, merged attention etc). Not sure it is even hardware scarcity per se / solely, it would just be really expensive in terms of both memory and FLOPs (and not clearly increase model capacity) to use larger matrices. Also for the specific part where you, in code for encoder-decoder transformers, call the a(x, x, y) function instead of the usual a(x, x, x) attention call (what Alammar calls "encoder-decoder attention" in his diagram just before the "The Decoder Side"), you have different matrix sizes, so dimension reduction is needed to make the matrix multiplications work out nicely too. But in general it is just a compute thing IMO.
  - tayo42 10mo ago
    >the terms "Query" and "Value" are largely arbitrary and meaningless in practice This is the most confusing thing about it imo. Those words all mean something but they're just more matrix multiplications. Nothing was being searched for.
    - D-Machine 10mo ago
      Better resources will note the terms are just historical and not really relevant anymore, and just remain a naming convention for self-attention formulas. IMO it is harmful to learning and good pedagogy to say they are anything more than this, especially as we better understand the real thing they are doing is approximating feature-feature correlations / similarity matrices, or perhaps even more generally, just allow for multiplicative interactions (https://openreview.net/forum?id=rylnK6VtDH https://openreview.net/forum?id=rylnK6VtDH).
- krat0sprakhar 10mo ago
  Do you have a tutorial that I can follow?
  - roadside_picnic 10mo ago
    The most valuable tutorial will be translating from the paper itself. The more hand holding you have in the process, the less you'll be learning conceptually. The pure manipulation of matrices is rather boring and uninformative without some context. I also think the implementation is more helpful for understanding the engineering work to run these models that getting a deeper mathematical understanding of what the model is doing.
  - jwitthuhn 10mo ago
    If you have 20 hours to spare I highly recommend this youtube playlist from Andrej Karpathy https://www.youtube.com/watch?v=VMj-3S1tku0&list=PLAqhIrjkxbuWI23v9cThsA9GvCAUhRvKZ https://www.youtube.com/watch?v=VMj-3S1tku0&list=PLAqhIrjkxb... It starts with the fundamentals of how backpropagation works then advances to building a few simple models and ends with building a GPT-2 clone. It won't taech you everything about AI models but it gives you a solid foundation for branching out.
- roadside_picnic 10mo ago
  I personally don't think implementation is as enlightening as far as really understanding what the model is doing as this statement implies. I had done that many times, but it wasn't until reading about the relationship to kernel methods that it really clicked for me what is really happening under the hood. Don't get me wrong, implementing attention is still great (and necessary), but even with something as simple as linear regression, implementing it doesn't really give you the entire conceptual model. I do think implementation helps to understand the engineering of these models, but it still requires reflection and study to start to understand conceptually why they are working and what they're really doing (I would, of course, argue I'm still learning about linear models in that regard!)
leopd 10mo ago
I think this video does a pretty good job explaining it, starting about 10:30 minutes in: https://www.youtube.com/watch?v=S27pHKBEp30 https://www.youtube.com/watch?v=S27pHKBEp30
- oofbey 10mo ago
  As the first comment says "This aged like fine wine". Six years old, but the fundamentals haven't changed.
- andoando 10mo ago
  This wasn't any better than other explanation I've seen.
throw310822 10mo ago
Have you tried asking e.g. Claude to explain it to you? None of the usual resources worked for me, until I had a discussion with Claude where I could ask questions about everything that I didn't get.
- sakesun 10mo ago
  Perhaps we have already reached ASI. :)
  - throw310822 10mo ago
    In some respects, yes. There is no single human being with a general knowledge as vast as that of a SOTA LLM, or able to speak as many languages. Claude knows about transformers more than enough to explain them to a layperson, elucidating specific points and resolving doubts. As someone who learns more easily by prodding other people's knowledge rather than from static explanations, I find LLMs extremely useful.
bobbyschmidd 10mo ago
tldr: recursively aggregating packing/unpacking 'if else if (functions)/statements' as keyword arguments that (call)/take them themselves as arguments, with their own position shifting according to the number "(weights)" of else if (functions)/statements needed to get all the other arguments into (one of) THE adequate orders. the order changes based on the language, input prompt and context. if I understand it all correctly. implemented it in html a while ago and might do it in htmx sometime soon. transformers are just slutty dictionaries that Papa Roach and kage bunshin no jutsu right away again and again, spawning clones and variations based on requirements, which is why they tend to repeat themselves rather quickly and often. it's got almost nothing to do with languages themselves and requirements and weights amount to playbooks and DEFCON levels
roadside_picnic 10mo ago
It's just a re-invention of kernel smoothing. Cosma Shalizi has an excellent write up on this [0]. Once you recognize this it's a wonderful re-framing of what a transformer is doing under the hood: you're effectively learning a bunch of sophisticated kernels (though the FF part) and then applying kernel smoothing in different ways through the attention layers. It makes you realize that Transformers are philosophically much closer to things like Gaussian Processes (which are also just a bunch of kernel manipulation). 0. http://bactra.org/notebooks/nn-attention-and-transformers.html http://bactra.org/notebooks/nn-attention-and-transformers.ht...
machinationu 10mo ago
Q, K and V are a way of filtering the relevant aspects for the task at hand from the token embeddings. "he was red" - maybe color, maybe angry, the "red" token embedding carries both, but only one aspect is relevant for some particular prompt. https://ngrok.com/blog/prompt-caching/ https://ngrok.com/blog/prompt-caching/
oedemis 10mo ago
there is also very good explanation from Luis Serrano, https://youtu.be/fkO9T027an0 https://youtu.be/fkO9T027an0

laser9 10mo ago

Here's the comment from the author himself (jayalammar) talking about other good resources on learning Transformers: https://news.ycombinator.com/item?id=35990118 https://news.ycombinator.com/item?id=35990118

boltzmann_ 10mo ago

Kudos also to Transformer Explainer team for putting some amazing visualizations https://poloclub.github.io/transformer-explainer/ https://poloclub.github.io/transformer-explainer/ It really clicked to me after reading this two and watching 3blue1brown videos

gzer0 10mo ago
This is hands down one of the best visualizations I have ever come across.

gustavoaca1997 10mo ago

I have this book. Really a life savior to help me catching up a few months ago when my team decided to use LLMs in our systems.

qoez 10mo ago
Don't really see why you'd need to understand how the transformer works to do LLMs at work. LLMs is just a synthetic human performing reasoning with some failure modes that in-depth knowledge of the transformer interals won't help you predict what they are (just have to use experience with the output to get a sense, or other peoples experiments).
- roadside_picnic 10mo ago
  In my experience this is a substantial difference in the ability to really get performance in LLM related engineering work from people who really understand how LLMs work vs people who think it's a magic box. If your mental model of an LLM is: > a synthetic human performing reasoning You are severely overestimating the capabilities of these models and not realizing potential areas of failure (even if your prompt works for now in the happy case). Understanding how transformers work absolutely can help debug problems (or avoid them in the first place). People without a deep understanding of LLMs also tend to get fooled by them more frequently. When you have internalized the fact that LLMs are literally optimistized to trick you, you tend to be much more skeptical of the initial results (which results in better eval suites etc). Then there's people who actually do AI engineering. If you're working with local/open weights models or on the inference end of things you can't just play around with an API, you have a lot more control and observability into the model and should be making use of it. I still hold that the best test of an AI Engineer, at any level of the "AI" stack, is how well they understand speculative decoding. It involves understanding quite a bit about how LLMs work and can still be implemented on a cheap laptop.
  - amelius 10mo ago
    But that AI engineer who is implementing speculative decoding is still just doing basic plumbing that has little to do with the actual reasoning. Yes, he/she might make the process faster, but they will know just as little about why/how the reasoning works as when they implemented a naive, slow version of the inference.

Koshkin 10mo ago

(Going on a tangent.) The number of transformer explanations/tutorials is becoming overwhelming. Reminds me of monads (or maybe calculus). Someone feels a spark of enlightenment at some point (while, often, in fact, remaining deeply confused), and an urge to share their newly acquired (mis)understanding with a wide audience.

nospice 10mo ago
So? There's no rule that the internet is limited to a single explanation. Find the one that clicks for you, ignore the rest. Whenever I'm trying to learn about concepts in mathematics, computer science, physics, or electronics, I often find that the first or the "canonical" explanation is hard for me to parse. I'm thankful for having options 2 through 10.
kadushka 10mo ago
Maybe so, but this particular blog post was the first and is still the best explanation of how transformers work.

ActorNightly 10mo ago

People need to get away from this idea of Key/Query/Value as being special. Whereas a standard deep layer in a network is matrix * input, where each row of the matrix is the weights of the particular neuron in the next layer, a transformer is basically input* MatrixA, input*MatrixB, input*MatrixC (where vector*matrix is a matrix), then the output is C*MatrixA*MatrixB*MatrixC. Just simply more dimensions in a layer. And consequently, you can represent the entire transformer architecture with a set of deep layers as you unroll the matricies, with a lot of zeros for the multiplication pieces that are not needed. This is a fairly complex blog but it shows that its just all matrix multiplication all the way down. https://pytorch.org/blog/inside-the-matrix/ https://pytorch.org/blog/inside-the-matrix/.

throw310822 10mo ago
I might be completely off road, but I can't help thinking of convolutions as my mental model for the K Q V mechanism. Attention has the same property of a convolution kernel of being trained independently of position; it learns how to translate a large, rolling portion of an input to a new "digested" value; and you can train multiple ones in parallel so that they learn to focus on different aspects of the input ("kernels" in the case of convolution, "heads" in the case of attention).
- krackers 10mo ago
  I think there are two key differences though: 1) Attention doesn't doesn't use fixed distance-dependent weight for the aggregation but instead the weight becomes "semantically-dependent", based on association between q/k. 2) A single convolution step is a local operation (only pulling from nearby pixels), whereas attention is a "global" operation, pulling from the hidden states of all previous tokens. (Maybe sliding window attention schemes muddy this distinction, but in general the degree of connectivity seems far higher). There might be some unifying way to look at things though, maybe GNNs. I found this talk [1] and at 4:17 it shows how convolution and attention would be modeled in a GNN formalism [1] https://www.youtube.com/watch?v=J1YCdVogd14 https://www.youtube.com/watch?v=J1YCdVogd14
  - 10mo ago

deleted 10mo ago

[deleted]

zkmon 10mo ago

I think the internal of transformers would become less relevant like internal of compilers, as programmers would only care about how to "use" them instead of how to develop them.

rvz 10mo ago
Their internals are just as relevant (now even more relevant) as any other technology as they always need to be improved to the SOTA (state of the art) meaning that someone has to understand their internals. It also means more jobs for the people who understand them at a deeper level to advance the SOTA of specific widely used technologies such as operating systems, compilers, neural network architectures and hardware such as GPUs or TPU chips. Someone has to maintain and improve them.
esafak 10mo ago
Practitioners already do not need to know about it to run let alone use LLMs. I bet most don't even know the fundamentals of machine learning. Hands up if you know bias from variance...
crystal_revenge 10mo ago
Have you written a compiler? I ask because for me writing a compiler was absolutely an inflection point in my journey as a programmer. Being able to look at code and reason about it all the way down to bytecode/IL/asm etc absolutely improved my skill as a programmer and ability to reason about software. For me this was the first time I felt like a real programmer.
- zkmon 10mo ago
  Writing a compiler is not a requirement or good use of time for a programmer. Same as why driving a car should not require you to build the car engine. Driver should stick to their role and learn how to drive properly.
  - crystal_revenge 10mo ago
    I'm guessing the answer then is "no". That's a ridiculous metaphor as well because building a compiler is a massive software engineering project that covers a huge range of essential skills. That metaphor would work for building a computer, but not a compiler. Clearly it shouldn't be a requirement, but it is an excellent use of a programmer's time. I can think of no software project over my career that has improved my skills more than writing a compiler.

libraryofbabel 10mo ago

I read this article back when I was learning the basics of transformers; the visualizations were really helpful. Although in retrospect knowing how a transformer works wasn't very useful at all in my day job applying LLMs, except as a sort of deep background for reassurance that I had some idea of how the big black box producing the tokens was put together, and to give me the mathematical basis for things like context size limitations etc. I would strongly caution anyone who thinks that they will be able to understand or explain LLM behavior better by studying the architecture closely. That is a trap. Big SotA models these days exhibit so much nontrivial emergent phenomena (in part due to the massive application of reinforcement learning techniques) that give them capabilities very few people expected to ever see when this architecture first arrived. Most of us confidently claimed even back in 2023 that, based on LLM architecture and training algorithms, LLMs would never be able to perform well on novel coding or mathematics tasks. We were wrong. That points towards some caution and humility about using network architecture alone to reason about how LLMs work and what they can do. You'd really need to be able to poke at the weights inside a big SotA model to even begin to answer those kinds of questions, but unfortunately that's only really possible if you're a "mechanistic interpretability" researcher at one of the major labs. Regardless, this is a nice article, and this stuff is worth learning because it's interesting for its own sake! Right now I'm actually spending some vacation time implementing a transformer in PyTorch just to refresh my memory of it all. It's a lot of fun! If anyone else wants to get started with that I would highly recommend Sebastian Raschka's book and youtube videos as way into the subject: https://github.com/rasbt/LLMs-from-scratch https://github.com/rasbt/LLMs-from-scratch . Has anyone read TFA author Jay Alammar's book (published Oct 2024) and would they recommend it for a more up-to-date picture?

nrhrjrjrjtntbt 10mo ago
It is almost like understanding wood at a molecular level and being a carpenter. It also may help the carpentery, but you cam be a great one without it. And a bad one with the knowledge.
ozgung 10mo ago
I think the biggest problem is that most tutorials use words to illustrate how the attention mechanism works. In reality, there are no word-associated tokens inside a Transformer. Tokens != word parts. An LLM does not perform language processing inside the Transformer blocks, and a Vision Transformer does not perform image processing. Words and pixels are only relevant at the input. I think this misunderstanding was a root cause of underestimating their capabilities.

some_guy_nobel 10mo ago

Great article, must be the inspiration for the recent Illustrated Evo 2: https://research.nvidia.com/labs/dbr/blog/illustrated-evo2/ https://research.nvidia.com/labs/dbr/blog/illustrated-evo2/

prashant418 10mo ago

This guide is such a beast, Try pairing this guide with say claude code and ask it to generate sample mini pytorch pesudo-code and you can spend hours just learning/re-learning and mentally visualize a lot of these concepts. I am a big fan

edge17 10mo ago

Maybe I'm out of touch, but have transformers replaced all traditional deep learning architectures? (U-nets, etc)?

D-Machine 10mo ago
No, not at all. There is a transformer obsession that is quite possibly not supported by the actual facts (CNNs can still do just as well: https://arxiv.org/abs/2310.16764 https://arxiv.org/abs/2310.16764), and CNNs definitely remain preferable for smaller and more specialized tasks (e.g. computer vision on medical data). If you also get into more robust and/or specialized tasks (e.g. rotation invariant computer vision models, graph neural networks, models working on point-cloud data, etc) then transformers are also not obviously the right choice at all (or even usable in the first place). So plenty of other useful architectures out there.
- edge17 10mo ago
  Is there something I can read to get a better sense of what types of models are most suitable for which problems? All I hear about are transformers nowadays, but what are the types of problems for which transformers are the right architecture choice?
  - D-Machine 10mo ago
    Just do some basic searches on e.g. Google Scholar for your task (e.g. "medical image segmentation", "point cloud segmentation", "graph neural networks", "timeseries classification", "forecasting") or task modification (e.g. "'rotation invariant' architecture") or whatever, sort by year, make sure to click on papers that have a large number of citations, and start reading. You will start to get a feel for domains or specific areas where transformers are and are not clearly the best models. Or just ask e.g. ChatGPT Thinking with search enabled about these kinds of things (and then verify the answer by going to the actual papers). Also check HuggingFace and other model hubs and filter by task to see if any of these models are available in an easy-to-use format. But most research models will only be available on GitHub somewhere, and in general you are just deciding between a vision transformer and the latest convolutional model (usually a ConvNext vX for some X). In practice, if you need to work with the kind of data that is found online, and don't have a highly specialized type of data or problem, then you do, today, almost always just want some pre-trained transformer. But if you actually have to (pre)train a model from scratch on specialized data, in many cases you will not have enough data or resources to get the most out of a transformer, and often some kind of older / simpler convolutional model is going to give better performance at less cost. Sometimes in these cases you don't even want a deep-learner at all, and just classic ML or algorithms are far superior. A good example would be timeseries forecasting, where embarrassingly simple linear models blow overly-complicated and hugely expensive transformer models right out of the water (https://arxiv.org/abs/2205.13504 https://arxiv.org/abs/2205.13504). Oh, right, and unless TabPFNv2 (https://www.nature.com/articles/s41586-024-08328-6 https://www.nature.com/articles/s41586-024-08328-6) makes sense for your use-case, you are still better off using boosted decision trees (e.g. XGBoost, LightGBM, or CatBoost) for tabular data.

Simplita 10mo ago

Visual explanations like this make it clearer why models struggle once context balloons. In practice, breaking problems into explicit stages helped us more than just increasing context length.

deleted 10mo ago

[deleted]

bearsortree 10mo ago

i found this much more intuitive to follow, https://poloclub.github.io/transformer-explainer/ https://poloclub.github.io/transformer-explainer/