3 ms·
If that's an issue, there's a workaround using structure generation to force it to output a </thiking> token after some threshold and force it to write the fina
by lelag 2y ago
If that's an issue, there's a workaround using structure generation to force it to output a </thiking> token after some threshold and force it to write the final answer.
It's a method used to control thinking token generation showcased in this paper: https://arxiv.org/abs/2501.19393 https://arxiv.org/abs/2501.19393