3 ms·

MoE models are harder to fine tune, and they don't solve the biggest problem which is GPU memory use.