When I wrote “backend” was a poor choice of a word. “Meta-model” is probably a better choice of wording. I hope it did not detract too much from the point of f

When I wrote “backend” was a poor choice of a word. “Meta-model” is probably a better choice of wording. I hope it did not detract too much from the point of focusing on subtasks and modalities for FOSS as GPT 4 was built on a $163 million budget. Finally, good point. We’ve got no idea of what OpenAI’s MoE approach is and how it works. I went back to Metas 2022 NLLB-200 system paper and they didn’t even publish the exact details of the router (gate).