3 ms·
Deepseek's DSpark does dynamically adjust speculated token count per user/completion. https://arxiv.org/abs/2607.05147 https://arxiv.org/abs/2607.05147 But th
by petu 2mo ago
Deepseek's DSpark does dynamically adjust speculated token count per user/completion.
https://arxiv.org/abs/2607.05147 https://arxiv.org/abs/2607.05147
But they do so to maximise total throughput, I don't think there's reason to do that for batch=1.