Close, fine tuning LLaMA in 8bit has been done. It requires very little VRAM compared to typical fine tuning. https://github.com/zphang/minimal-llama#peft-fine

Close, fine tuning LLaMA in 8bit has been done. It requires very little VRAM compared to typical fine tuning. https://github.com/zphang/minimal-llama#peft-fine-tuning-with-8-bit https://github.com/zphang/minimal-llama#peft-fine-tuning-wit...