3 ms·
What’s the best source to understand how gpt3 works? Ideally dumbed down a bit for a lay person.
by mrfusion 6y ago
What’s the best source to understand how gpt3 works? Ideally dumbed down a bit for a lay person.
- camjohnson26 6y agoThis seems reasonable. The core mechanism is a “transformer” https://jalammar.github.io/illustrated-gpt2/ https://jalammar.github.io/illustrated-gpt2/ GPT3 is essentially GPT2 with more training data.
- sjg007 6y agoRead the attention is all you need paper. Then read the gpt2 paper. Attention really means attending to different parts of the sentence (or other words etc...) There are a couple of tech talks on YouTube that help. Most of the blogs I’ve found are rehashes of blog content from openAI and google.