4 ms·
They already support quantization out of the box! https://github.com/huggingface/candle/pull/314/files https://github.com/huggingface/candle/pull/314/files
by FL33TW00D 3y ago
They already support quantization out of the box!
https://github.com/huggingface/candle/pull/314/files https://github.com/huggingface/candle/pull/314/files