2 ms·
GGML hasn't been a thing for some time, and GGUF (its successor) has features such as "importance matrix" quantization that is all about quantizing adaptively.
by int_19h 1y ago
GGML hasn't been a thing for some time, and GGUF (its successor) has features such as "importance matrix" quantization that is all about quantizing adaptively. Then there's all the stuff that Unsloth does, e.g.: https://unsloth.ai/blog/dynamic-v2 https://unsloth.ai/blog/dynamic-v2