3 ms·
I'm seeing this infinite loop behavior during reasoning as well, for the exact same quant from Unsloth. Set presence penalty to 1 and the issue seemly went away
by ssvegeta 2mo ago
I'm seeing this infinite loop behavior during reasoning as well, for the exact same quant from Unsloth. Set presence penalty to 1 and the issue seemly went away, but having a penalty that high worries me for coding tasks. Will try the bartowski one now.