3 ms·
How big is GPT4o-mini? Some sources say it's 8b big, but I guess they have different models with different sizes. But if GPT4o-mini is just 8b, I don't see the
by sinuhe69 2y ago
How big is GPT4o-mini? Some sources say it's 8b big, but I guess they have different models with different sizes. But if GPT4o-mini is just 8b, I don't see the point of a "distilled" model, which requires a much bigger network but still not on par with the original. Because it's open source?
- danielhanchen 2y agoI think it's a MoE probably with 8b activated parameters - so not exactly 8b but maybe 8b x 8 But on many benchmarks it does surpass GPT 4o mini - some benchmarks are even better than GPT 4o. But yes in general it's because its a powerful open source models!