3 ms·
> This leads to output that we would classify as "very boring". Not really. I set temperature to 0 for my local models, it works fine. The reason why the clou
by otabdeveloper4 1y ago
> This leads to output that we would classify as "very boring".
Not really. I set temperature to 0 for my local models, it works fine.
The reason why the cloud UIs don't allow a temperature of 0 is because then models sometimes start to do infinite loops of tokens, and that would break the suspension of disbelief if the public saw it.
- mdp2021 1y agoWhich local models are you using, that do not output loop garbage at temperature 0? What do you get at very low temperature values instead of 0?
- otabdeveloper4 1y ago> Which local models are you using, that do not output loop garbage at temperature 0? All of them. I make my own frontends using llama-cpp. Quality goes up with temperature 0 and loops are rare. The temperature setting isn't for improving quality, it's to not break your suspension of disbelief that you're talking to an intelligent entity.
- mdp2021 1y ago> All of them You must be using recent (or just different) models than those I tried. Mine returned garbage easily at temperature 0. (But unfortunately, I cannot try and report from there.) This (LLM behaviour and benchmarking at low or 0 temperature value) should be a topic to investigate.
- otabdeveloper4 1y agoProbably a bug in the code you ran somewhere.