3 ms·
Yes. The actual output is all the tokens and a probability for each. You can pick the highest or take one randomly using those probabilities. Depends how creati
by quickthrower2 3y ago
Yes. The actual output is all the tokens and a probability for each. You can pick the highest or take one randomly using those probabilities. Depends how creative or deterministic you want it to be.
- cubefox 3y agoExactly. A temperature of 0 means you always pick the highest probability token (i.e. the "max" function), while a temperature of 1 means you randomly pick a token according to their given probability. Values in between are also possible. However, it's important to note that the numbers which the models returns for the next tokens can only be interpreted as probabilities for foundation models. For fine-tuned models (instruction SL, RLHF) the numbers represent how good the model judges the next token to be. This also leads to a phenomenon called mode collapse.
- quickthrower2 3y agoVery interesting to know! (about FM vs FTM) Thanks.