4 ms·
Because the model (=="dictionary") is 70B floats -- 280GB naively, 40-70GB aggressively quantized (which might reduce compression rate). If your file is big eno
by jmoss20 3y ago
Because the model (=="dictionary") is 70B floats -- 280GB naively, 40-70GB aggressively quantized (which might reduce compression rate). If your file is big enough that the marginal compression win over other methods makes this space-effective, sure. But that's a very narrow case.