3 ms·

The article linked here talks about GPT4 being a mixture of experts, which is exactly what you’re describing