3 ms·
isn't that what the mixture of experts trick that all the big players do is? Bunch of smaller, tightly focused models
by agoodusername63 11mo ago
isn't that what the mixture of experts trick that all the big players do is? Bunch of smaller, tightly focused models
- irthomasthomas 10mo agoNot exactly. MoE uses a router model to select a subset of layers per token. This makes them faster but still requires the same amount of RAM.