4 ms·
The temperature parameter doesn’t introduce any noise into the network evaluation. Typically, what happens is that the network outputs a set of possible tokens
by Filligree 2y ago
The temperature parameter doesn’t introduce any noise into the network evaluation.
Typically, what happens is that the network outputs a set of possible tokens with different probabilities, and a sampler picks from the top possibilities. Temperature determines how spiky its pick is; at zero it’ll always pick the top option.