The output is a probability distribution for all potential tokens. Then a "temperature" is applied to weight the sampling randomly (unless the temperature is z

The output is a probability distribution for all potential tokens. Then a "temperature" is applied to weight the sampling randomly (unless the temperature is zero in which case the stack can be deterministic and simply the highest probability token is selected). They go through this rigamarole because a little bit of randomness gives better results from a Turing test kind of perspective.