Read the attention is all you need paper. Then read the gpt2 paper. Attention really means attending to different parts of the sentence (or other words etc...

Read the attention is all you need paper. Then read the gpt2 paper. Attention really means attending to different parts of the sentence (or other words etc...) There are a couple of tech talks on YouTube that help. Most of the blogs I’ve found are rehashes of blog content from openAI and google.