3 ms·

Mixture of Experts. Very broadly speaking, there are a bunch of mini networks (experts) which can be independently activated.