4 ms·
Yes, the GPT's are literally trained to minimize (cross-)entropy of text. You can see the bits-per-byte of text for lots of models here: https://crfm.stanford.e
by canjobear 3y ago
Yes, the GPT's are literally trained to minimize (cross-)entropy of text. You can see the bits-per-byte of text for lots of models here: https://crfm.stanford.edu/helm/v0.2.0/?group=targeted_evaluations https://crfm.stanford.edu/helm/v0.2.0/?group=targeted_evalua...