2 ms·

Can someone please enlighten me how this is any different from Mixture of Experts? Because I don't see any difference at all.

• liuliu 2y ago
  The router is manually designed (see their cem function). Also, the experts are not separate weights, just different scales of it's singular values.
  • ghc 2y ago
    Thank you, I was missing that second part.