3 ms·
Can you do tuning/training on quantized models? Or are they inference only?
by lawlessone 3y ago
Can you do tuning/training on quantized models? Or are they inference only?
- woadwarrior01 3y agoThis paper[1] attempts to do LoRA fine tuning on quantised LLMs. The LoRa weights themselves are full precision, but they're much smaller. Although, I haven't had a chance to try it yet. [1]: https://arxiv.org/abs/2306.08162 https://arxiv.org/abs/2306.08162