3 ms·
It wasn't either/or, the N-gram table is part of Qwen itself and stays on disk. I’ve now added its 1.5GB MTP draft head too, it gets 86% acceptance and about 1.
by carloslfu 1mo ago
It wasn't either/or, the N-gram table is part of Qwen itself and stays on disk. I’ve now added its 1.5GB MTP draft head too, it gets 86% acceptance and about 1.24× faster decoding on my 48GB Mac.
- siris9476 1mo agoGot it, thanks for clearing that up. 86% acceptance is solid — does it stay flat over longer generations, or drift with context length?