As I understand it, LLMs use temperature scaling to optimize the variability in the tokens they generate so the output appears "good", somewhere between getting

As I understand it, LLMs use temperature scaling to optimize the variability in the tokens they generate so the output appears "good", somewhere between getting stuck always saying the same thing and saying nonsense. [0] I wonder if more output being used in the training data will result in a shift to higher temperatures over time to keep variety in the output. [0] Discussion the other day: https://news.ycombinator.com/item?id=35131112 https://news.ycombinator.com/item?id=35131112