2 ms·
Looping is a common problem with the Qwen models. I've had good luck using --repeat-penalty=1.1 with llama.cpp and 27B. vLLM should have a similar option.
by cpburns2009 4mo ago
Looping is a common problem with the Qwen models. I've had good luck using --repeat-penalty=1.1 with llama.cpp and 27B. vLLM should have a similar option.
- etdznots 4mo agoThis is the default value!
- cpburns2009 4mo agoLlama.cpp defaults to 1.0 (disabled) and so does vLLM. It looks like only ollama defaults to 1.1.
- Der_Einzige 4mo agoPlease switch to using the far superior reptation penalty, DRY. It's built into llamacpp.
- cpburns2009 4mo agoWhat are good DRY setting