3 ms·
From https://huggingface.co/Qwen/QwQ-32B https://huggingface.co/Qwen/QwQ-32B Presently, vLLM only supports static YARN, which means the scaling factor remains
by smallerize 2y ago
From https://huggingface.co/Qwen/QwQ-32B https://huggingface.co/Qwen/QwQ-32B
Presently, vLLM only supports static YARN, which means the scaling factor remains constant regardless of input length, potentially impacting performance on shorter texts. We advise adding the rope_scaling configuration only when processing long contexts is required.
- GTP 2y agoSorry, could you please explain what this means? I'm not into machine learning, so I don't get the jargon.
- smallerize 2y agoWell I can't be positive, but it looks like some of the factors that support a long context length might be set wrong. https://blog.eleuther.ai/yarn/ https://blog.eleuther.ai/yarn/