Size reduction is mostly based on Experts size. And it is limited by SSD speed. Check for Colibri and Flash-Moe, they are doing similar things with bigger model

Size reduction is mostly based on Experts size. And it is limited by SSD speed. Check for Colibri and Flash-Moe, they are doing similar things with bigger models, but tok/s is not high