The actual title "Why the Original Transformer Figure Is Wrong, and Some Other Interesting Historical Tidbits About LLMs" is way more representative of what thi

The actual title "Why the Original Transformer Figure Is Wrong, and Some Other Interesting Historical Tidbits About LLMs" is way more representative of what this post is about... As to the figure being wrong, it's kind of a nit-pick: "While the original transformer figure above (from Attention Is All Your Need, https://arxiv.org/abs/1706.03762 https://arxiv.org/abs/1706.03762) is a helpful summary of the original encoder-decoder architecture, there is a slight discrepancy in this figure. For instance, it places the layer normalization between the residual blocks, which doesn't match the official (updated) code implementation accompanying the original transformer paper. The variant shown in the Attention Is All Your Need figure is known as Post-LN Transformer."