7 ms·
What it tells you is that 93% of the information is sufficiently shared with the rest of the English language such that it can be pulled out into a shared codeb
by regularfry 3y ago
What it tells you is that 93% of the information is sufficiently shared with the rest of the English language such that it can be pulled out into a shared codebook. LZMA doesn't have a codebook, not really.
In other words it's not that llama2 contains 93% of Chapter 1, it's that only 7% of Chapter 1 is different enough to anything else to be worth encoding in its own right.