3 ms·
This topic has been reposted few times recently, yet never gained much traction. I wonder how much changes there have been between GPT2 to GPT4?
by 3abiton 3y ago
This topic has been reposted few times recently, yet never gained much traction. I wonder how much changes there have been between GPT2 to GPT4?
- kridsdale1 3y agoMixture of Experts model is likely the most significant. And the scale of everything. GPT3 embedding vectors are around 12,000, vs 768 shown here. I was curious and the 12k figure closely approximates the median synapse dimensionality of human neurons. Maybe we don’t need much more.