3 ms·
It's not explicitly mentioned, but they do implicitly say it here: > We then apply the softmax function to convert the logits into a probability distribution t
by magicalhippo 2y ago
It's not explicitly mentioned, but they do implicitly say it here:
> We then apply the softmax function to convert the logits into a probability distribution that sums to one.
To go from a probability distribution to a single token, you need to sample it, ie pick a random element according to the probability distribution[1]. Elements with higher probability have a higher chance to get picked, but it's still random. This is also why the "temperature" parameter works the way it does.
[1]: https://en.wikipedia.org/wiki/Non-uniform_random_variate_generation#Finite_discrete_distributions https://en.wikipedia.org/wiki/Non-uniform_random_variate_gen...
- jebarker 2y agoAh, I guess the examples are canned for efficiency since it picks the same token each time. Thanks for clarifying.
- magicalhippo 2y agoHmm, if I select the same example multiple times from the drop-down, it does pick a different output value each time. So seems it does implement the sampling "properly" as such.
- jebarker 2y agoYou're right - if you select it from the drop-down it works that way. If you refresh the browser, as I was, it defaults to the same sample.
- magicalhippo 2y agoAh yes, probably using a fixed seed value for the pseudo-random generator then, so the sequence is fixed as such, but the individual numbers are not correlated. Not unusual.