27 ms·Given how "smart" some of the 26b dense models are now, I would not be surprised to see a strong 40b MoE.by bearjaws 4mo agoGiven how "smart" some of the 26b dense models are now, I would not be surprised to see a strong 40b MoE.