3 ms·
Yes, you can think of this as a collection of small models, with another model choosing which smaller model to use for each input.
by p1esk 6y ago
Yes, you can think of this as a collection of small models, with another model choosing which smaller model to use for each input.
- gwenzek 6y agoNot really, the dispatching between experts happen for every word and every layer, so you can't easily isolate distinct models.