The quoted paper yes, but the MoE concept and layers and training is old. Published as a conference paper at ICLR 2017 OUTRAGEOUSLY LARGE NEURAL NETWORKS: THE

The quoted paper yes, but the MoE concept and layers and training is old. Published as a conference paper at ICLR 2017 OUTRAGEOUSLY LARGE NEURAL NETWORKS: THE SPARSELY-GATED MIXTURE-OF-EXPERTS LAYER Noam Shazeer, Azalia Mirhoseini, Krzysztof Maziarz, Andy Davis, Quoc Le, Geoffrey Hinton and Jeff Dean