4 ms·
In practice, it is even less. Current state-of-the-art algorithms compress the English Wikipedia using just 0.8 bits per character: https://www.mattmahoney.net/
by taffer 2y ago
In practice, it is even less. Current state-of-the-art algorithms compress the English Wikipedia using just 0.8 bits per character: https://www.mattmahoney.net/dc/text.html https://www.mattmahoney.net/dc/text.html
- esperent 2y agoWhy do you think Wikipedia is an accurate representation of the English language? Is it just because it's large? As an encyclopedia, it has an intentionally limited and factual way of describing things, which lacks a lot of important parts of language like poetry, allegory, metaphor, slang, regional dialects, and so on. The fact that it can be compressed down so much probably just means it has a ton of repetition.