9 ms·

All modern models are MoE already, no?

• hasperdi 11mo ago
  That's not the case. Some are dense and some are hybrid. MOE is not the holy grail, as there are drawbacks eg. less consistency, expert under/over-use