4 ms·
Show HN: LLM Attention Visualization
- sva_ 26d agoI highly question this simplistic idea of high vector magnitude = high influence.
- ifz 26d agoI don't disagree with that. I did add an entire caveat paragraph there. To me, it's more of a neat visualization, not something that can be used to interpret LLM behavior. Even with a lot of simplification, it can show some interesting patterns.
- smallmancontrov 26d agoYou get what you pay for. If you want to think harder and get more, https://transformer-circuits.pub/2025/attention-qk/index.html https://transformer-circuits.pub/2025/attention-qk/index.htm...
- apnabhidu47 26d agoSame I dont get it just, could you clarify it
- colophontio 26d ago[dead]
- itsnasme 26d agoI like the visualisation. Pretty cool
- wopak 26d agoneat, combining info from two phrases is hard to see without such a tool. are you worried later-layer attention gets drowned out by earlier layers just because there are more of them contributing to the sum?
- ifz 26d agoHmm, I might try to add some controls to limit which layers get summed up. It might be able to reveal more patterns. Right now only simple correlations are visible.
- stared 26d agoI am curious what's the actual formula. I mean, there so many headers and layers, it is tricky to make a choice that will resonate with our intuition . Is it some weighted average? Or maybe ablation test?
- ifz 26d agoIt's really simple, basically just the magnitude of the value vector, weighted by QK dot product, summed across all attention heads and layers. When I started, I expected I'd have to experiment a lot to find something comprehensible. But this simple computation can already show some patterns.
- stared 26d agoNice! Sometimes the simplest approaches work the best.
- nullbio 26d agoMight be cool to try different colors for the different attention heads instead of summing them across all attention heads, and making the backgrounds composed of stacked color layers? So you can see how each attention head attends to tokens individually.
- visarga 26d agoIf you want quick access look at google images for "transformer attention formula" there are some interesting depictions
- fuddle 26d agoThis is great, I've read multiple books and watched videos about the attention mechanism. Now that I understand it, this is the clearest example I've seen on how attention works.
- ex-aws-dude 26d agoI don't know much about LLMs but does that mean you have N^2 computation with the context size since every token needs to track how it relates to every other token?
- deleted 26d ago[deleted]
- acedTrex 26d agoFor full self attention yes
- TomatoCo 26d agoYes, except no with the KV cache. Because tokens aren't modified by future tokens you can cache the meaning of previous tokens. This makes the total effort linear over the entire context (or constant per forward pass).
- ex-aws-dude 26d agoI see and is there only 1 layer of relations? Or does it accumulate the relations like A relates to B, so also add in B's relations
- TomatoCo 24d agoWell, I'm under the impression that the relations are recursive but you skip computing them recursively by caching B's relations. However if you look at your sibling comments you'll see that I may not have the right grasp on the matter.
- libraryofbabel 26d ago> This makes the total effort linear over the entire context (or constant per forward pass). This is incorrect. The compute required per forward pass to generate each additional token during decode will scales as O(N), even with a KV cache (without a KV cache, it would scale as O(N^2)). Over generating N tokens, it's O(N^2) with the cache (and O(N^3) without). It's O(N) for a forward pass because that new token still has to "attend to" to each previous token. That requires N dot products: between the cached key vectors and the new query vector for the new position. You also have N reads from memory (K and V) which is probably gonna be your actual bottleneck. (Decode is memory-bound.) This is why you should avoid long contexts, if you can, even with a warm cache. You will get charged more, in "cache read" tokens.
- Yyylov 26d ago[dead]
- MCP123 26d agoThis is great, thank you. I have to teach this stuff on Friday so perfect timing. It's hard to explain the attention mechanism in a way that becomes intuitive because the weighting scheme does not help much with the intuition. Having a visualization like this helps a lot. Don't move that page please since I'll link to it!
- fermlon30000 26d agoINSANE
- mncharity 26d ago[dead]
- mncharity 26d agoUX report. I wished to examine attention state step by step, but I found the animation moved along too fast for that. So I tried pausing... On Chromium/linux, pressing pause doesn't pause, instead resetting the animation to it's pre-play state - the current attention highlighting disappears. Pressing play again, restarts at the beginning. Having a commonplace "pause pauses, and play resumes" UI, could allow more time to look over state. A youtube-like slow playback 0.25? option might similarly help. Or perhaps even better, buttons for single stepping. Tnx for your work.
- ifz 24d agoThanks for trying it out! You can hover / click / tap on any word to focus it. I actually intended for this to be main way to use the visualization, I just added the auto-playback later on so it's more eye-catching on page load.
- scottcodie 26d agoYou can also mine attention from image models, it's a lot of fun and very interesting.
- lhk931122 26d agoIs the attention explanation of why the model tells like this? I've seen that there are many discussions about this. (Image attention visualizations were not that good I think)
- asd000hh 26d agoHow it works?
- talhaanwar 26d agothanks for making it simple and visualizable
- asamoahf 25d ago[flagged]
- sshussain270 25d agoThis is very cool. It's simpler than Bertviz for understanding inference and surface level and a good starting people for new learners as well.
- ifz 24d agoOh, I actually didn't know about Bertviz before. That looks interesting as well!
- zephyroszhou 24d agoVery cool stuff.
- Mugshelf 23d agoThe pause-on-load problem is a real UX trap. Stepping through one token at a time is how people actually learn attention; single-step controls would help more than autoplay.