3 ms·
You unfold it one token at a time by sampling from the returned distribution. To control the amount of variation, you can make the probability distribution more
by ruuda 4y ago
You unfold it one token at a time by sampling from the returned distribution. To control the amount of variation, you can make the probability distribution more extreme, in the most extreme case you only select the most likely token, and the sequence becomes deterministic.
Yes, what happens later in the sentence depends on the particular choice you made earlier in the sentence.