3 ms·
Didn't it leak earlier that it is 100 billion? And GPT 4 is 1.17 Trillion?
by sidcool 3y ago
Didn't it leak earlier that it is 100 billion? And GPT 4 is 1.17 Trillion?
- nacs 3y agoThat 1 trillion number was quickly confirmed to be false. It's believed that GPT4 is a Mixture of Experts model -- it's 8 smaller models (estimated around 200B each) and a router picks the best model to redirect to based on input.
- hencoappel 3y agoSo 800B + a router network, so not far off a trillion?