2 ms·
Due to the default settings of `top_k`, `top_p`, and `temperature`, which introduce a probabilistic element to token selection, two runs of the same prompt with
by rahimnathwani 3y ago
Due to the default settings of `top_k`, `top_p`, and `temperature`, which introduce a probabilistic element to token selection, two runs of the same prompt with the same model may yield different outputs.
The first two parameters, `top_k` and `top_p`, define the subset of tokens eligible for selection, while `temperature` influences the probability distribution from which tokens are sampled.