This is not how MoEs work at all. They are all trained together, often you have multiple experts activated for a single token. They are not domain specific in a

This is not how MoEs work at all. They are all trained together, often you have multiple experts activated for a single token. They are not domain specific in any way that is understandable by humans.