3 ms·
They have not. You probably read "MoE" and some pop article about what that means without having any clue.
by whimsicalism 2y ago
They have not. You probably read "MoE" and some pop article about what that means without having any clue.
- matsemann 2y agoIf you know better it would be nice of you to provide the correct information, and not just refute things.
- whimsicalism 2y agogpt-4 is a sparse MoE model with ~1.2T params. this is all public knowledge and immediately precludes the two previous commentators assertions