3 ms·
I'm pretty sure he suggested it was a 16 way 110 MoE
by qeternity 3y ago
I'm pretty sure he suggested it was a 16 way 110 MoE
- brandall10 3y agoThe exact quote: "Sam Altman won’t tell you that GPT 4 has 220 billion parameters and is a 16 way mixture model with eight sets of weights."