Shit... I was just looking at the DeepSeekMoE paper, almost exactly from 1 year ago: https://arxiv.org/html/2401.06066v1 https://arxiv.org/html/2401.06066v1 T

Shit... I was just looking at the DeepSeekMoE paper, almost exactly from 1 year ago: https://arxiv.org/html/2401.06066v1 https://arxiv.org/html/2401.06066v1 The red star in the first chart is so funny.