3 ms·
It's included in the model gguf itself.
by greenicon 2mo ago
It's included in the model gguf itself.
- gaigalas 2mo agoMaybe I'm not seeing the speed improvements because it's a dense model, and I'm comparing it to the 3.6 MoE with MTP. Likely, I'm attributing the speed I see to something else. Honestly, if MTP is available, I don't know why it's so much slower here.