2 ms·
MoE models aren’t new. Rumors are that GPT4 was one.
by aurareturn 2y ago
MoE models aren’t new. Rumors are that GPT4 was one.
- swyx 2y agoyea specifically 8x220B https://www.latent.space/p/geohot https://www.latent.space/p/geohot
- enragedcacti 2y agoYes, with substantially more active params for SOTA.