That 1 trillion number was quickly confirmed to be false. It's believed that GPT4 is a Mixture of Experts model -- it's 8 smaller models (estimated around 200B

That 1 trillion number was quickly confirmed to be false. It's believed that GPT4 is a Mixture of Experts model -- it's 8 smaller models (estimated around 200B each) and a router picks the best model to redirect to based on input.