The infrastructure is almost there now. LLMs are already training small domain specific models. Small language models have already shown excellent performance i

The infrastructure is almost there now. LLMs are already training small domain specific models. Small language models have already shown excellent performance in specific domains, particularly logic and math. All that's needed is an extensible mixture of experts router so that these small models can be plugged into the larger model and selected when the query is appropriate. This isn't far off.