2 ms·
The point he is making is not consistent with understanding how temperature influences LLM text generation, no. He repeatedly states that choosing "the best wo
by wasabi991011 1mo ago
The point he is making is not consistent with understanding how temperature influences LLM text generation, no.
He repeatedly states that choosing "the best word" is the most important thing to him. I don't know how you reconcile that with creativity itself, let alone probabilistic sampling.
- dofm 1mo agoI mean creativity there in the LLM sense (temperature driving more creative solutions), not the human sense, and in the context of its consistency, configurability and being amenable to analysis. The watermarking approach makes that non-reproducible, yes? Because Anthropic can and will change it as they see fit.
- baobabKoodaa 1mo agoYour response seems to be missing the point completely. Gruber thinks "best" writing is produced by choosing the "best" word (highest scoring token) at each step. This is very clear from his writing.
- eru 1mo agoYes, naively understood in his sense 'best' word means that you pick the word with the maximum score, instead of sampling from the distribution. That doesn't actually give you the 'best' text in any human sense of the word. Just like playing the 'best' move in Poker without sampling leads you to lose a lot of money.