3 ms·
Isn't it not completely quantized? I thought there were some dense parts but most is int4?
by knollimar 4mo ago
Isn't it not completely quantized? I thought there were some dense parts but most is int4?
- wgd 4mo agoOften in MoE models the experts are quantized while the shared portions, being a much smaller part of the network with greater impact, are kept at higher or full precision. Not familiar with the Kimi QAT approach specifically but it's likely they do this.