Nobody decides. The network itself determines which expert(s) to activate based on the context. It uses a small neural network for the task. It typically wo

Nobody decides. The network itself determines which expert(s) to activate based on the context. It uses a small neural network for the task. It typically won't behave like human experts - you might find one of the networks is an expert in determining where to place capital letters or full stops for example. MoE's do not really improve accuracy - instead they are to reduce the amount of compute required. And, assuming you have a fixed compute budget, that in turn might mean you can make the model bigger to get better accuracy.