4 ms·
> If you want to understand “how GPT-3 works”, you can just study GPT-2. Nothing is different architecturally between them. This isn't true. At the very least,
by ianhorn 6y ago
> If you want to understand “how GPT-3 works”, you can just study GPT-2. Nothing is different architecturally between them.
This isn't true. At the very least, off the top of my head, GPT-2 didn’t have the sparse layers that GPT-3 has.