4 ms·
Paper: https://arxiv.org/abs/2306.04050 https://arxiv.org/abs/2306.04050 > We provide new estimates of an asymptotic upper bound on the entropy of English usin
by bcjordan 3y ago
Paper: https://arxiv.org/abs/2306.04050 https://arxiv.org/abs/2306.04050
> We provide new estimates of an asymptotic upper bound on the entropy of English using the large language model LLaMA-7B as a predictor for the next token given a window of past tokens.
> Preliminary results from limited experiments suggest that our scheme outperforms state-of-the-art text compression schemes such as BSC, ZPAQ, and paq8h.