4 ms·
Thank you for your contribution!!! Does min_p help with non-creative-writing tasks as well, such as maths or coding? Is there any way to improve/tune the perfo
by thegeomaster 1y ago
Thank you for your contribution!!!
Does min_p help with non-creative-writing tasks as well, such as maths or coding? Is there any way to improve/tune the performance for these with sampling?
Do you have a recommendation/guide on tuning the sampling parameters with what little the frontier model providers expose?
All I have ever seen is the same old very old advice of "t=0 for determinism, t=1 for creativity", with little treatment of top_k/top_p. But even the temp has become quite opaque nowadays. Is this really the softmax temperature that we're controlling, or some proxy for it? Claude 3.7 Sonnet doesn't allow me to use t != 1.0 for thinking mode, while Gemini 2.5 Pro, also a reasoner, will happily accept it. Does it apply only to the non-thinking tokens at the end?
If I get good results with t=0.0 (for e.g. coding), do I lose some "capability" by keeping it at 0.0?
Thanks in advance.