If I understand this correctly we have come full circle on what MoE means. MoE started out as some form of multi model approach. Afaik in current architecture

If I understand this correctly we have come full circle on what MoE means. MoE started out as some form of multi model approach. Afaik in current architectures it’s basically a load balancing method that while it increases latency makes the model better suitable for distributed operations. To me this reads as if the author uses the term closer to Urs original meaning than its current.