4 ms·
It's actually a 14.3B parameter model. It's irritating that they don't follow the convention of naming the model according to size. Qwen1.5-MoE-A2.7B is named
by kristianp 3y ago
It's actually a 14.3B parameter model. It's irritating that they don't follow the convention of naming the model according to size. Qwen1.5-MoE-A2.7B is named for the 2.7B activated parameters. I guess it helps to obfuscate the size, given that it performs about as well as Mistral 7B.