3 ms·
Decoding (sampling) uses (pseudo) random numbers. Otherwise same prompt would always give the same response. Computing entropy generally does not. See e.g. ht
by jampekka 2y ago
Decoding (sampling) uses (pseudo) random numbers. Otherwise same prompt would always give the same response.
Computing entropy generally does not.
See e.g. https://huggingface.co/blog/how-to-generate https://huggingface.co/blog/how-to-generate
- HarHarVeryFunny 2y agoSure - but that's not the output of the model itself, that's the process of (typically) randomly sampling from the output of the model.
- throwaway314155 2y agoRight, sampling from a model, also known as *inference* (for LLM's). The inference here is perhaps less pure than what you refer to but you're talking to human beings; there's no need for heavy pedantry.