3 ms·
These quantized models show much less degradation compared to a "vanilla post-training-quantization" but there are a bunch of PTQ schemes that people have alrea
by philipkglass 2y ago
These quantized models show much less degradation compared to a "vanilla post-training-quantization" but there are a bunch of PTQ schemes that people have already applied to Llama models [1]. I didn't see any details about the vanilla PTQ they used as a baseline. Has it been written about elsewhere?
[1] https://ollama.com/library/llama3.2/tags https://ollama.com/library/llama3.2/tags