Reading this could make people believe it is computed from the probability distribution of the model alone. To be clearer, it is the exponent of the average ne

Reading this could make people believe it is computed from the probability distribution of the model alone. To be clearer, it is the exponent of the average negative log probability that the model gives to the real tokens of a sample text[0]. Roughly, it relates to how strongly the model can predict the sample text. A perfect model would have perplexity one; a random model has a perplexity equal to the number of possible tokens; the worst model has infinite perplexity. [0]: https://github.com/pytorch/torcheval/blob/3faf19c060b8a7c074d76db0a998baa7a865c5ad/torcheval/metrics/functional/text/perplexity.py#L115 https://github.com/pytorch/torcheval/blob/3faf19c060b8a7c074...