4 ms·Doubling speed can likely come from MoE optimizations such as reducing the amount of active parameters.by AmazingTurtle 8mo agoDoubling speed can likely come from MoE optimizations such as reducing the amount of active parameters.