4 ms·
I mean, if you are attempting to use multiple sampling to avoid this kind of error, just use temperature 0 sampling and do it once. In your example you will ge
by spott 2y ago
I mean, if you are attempting to use multiple sampling to avoid this kind of error, just use temperature 0 sampling and do it once.
In your example you will get Pete every time.
- wokwokwok 2y agoSure, that's fair. I will say though, using temperature 0 without understanding it (or worse, testing at temp > 0 and then setting temp to 0 for production, which I literally had to stop someone I know and respect as a developer from doing) and not understanding what top_k and top_n do (but using them anyway) is my #3 for LLM fails. /shrug ...but, yes as you say, in a trivial case, like binary decision making, a 0 or very low temperature can help with the need to multiple sample; and as you say, when it's deterministic, sampling multiple times doesn't help at all.