4 ms·
The router is manually designed (see their cem function). Also, the experts are not separate weights, just different scales of it's singular values.
by liuliu 2y ago
The router is manually designed (see their cem function). Also, the experts are not separate weights, just different scales of it's singular values.
- ghc 2y agoThank you, I was missing that second part.