2 ms·
Sure, that's fair. I will say though, using temperature 0 without understanding it (or worse, testing at temp > 0 and then setting temp to 0 for production, wh
by wokwokwok 2y ago
Sure, that's fair.
I will say though, using temperature 0 without understanding it (or worse, testing at temp > 0 and then setting temp to 0 for production, which I literally had to stop someone I know and respect as a developer from doing) and not understanding what top_k and top_n do (but using them anyway) is my #3 for LLM fails.
/shrug
...but, yes as you say, in a trivial case, like binary decision making, a 0 or very low temperature can help with the need to multiple sample; and as you say, when it's deterministic, sampling multiple times doesn't help at all.