4 ms·
Deepseek is 651B, GPT models are suspected to be <200B. Where are they getting the idea that Deepseek is more efficient?
by HeatrayEnjoyer 2y ago
Deepseek is 651B, GPT models are suspected to be <200B. Where are they getting the idea that Deepseek is more efficient?
- enragedcacti 2y agoIts an MoE model so only 37B parameters are activated per token.
- aurareturn 2y agoMoE models aren’t new. Rumors are that GPT4 was one.
- swyx 2y agoyea specifically 8x220B https://www.latent.space/p/geohot https://www.latent.space/p/geohot
- enragedcacti 2y agoYes, with substantially more active params for SOTA.