4 ms·
Tim Dettmers: QLoRA finetunes a 65B model on a single 48 GB GPU
- convexstrictly 3y agoClosed beta runs for a week. 4 bit floats. Quantization on top of quantization. Finetune Alpaca 7B in ~3 hours on an A40 GPU/RTX3090. Paging of optimizer states. Uses LoRA. Integrated with HuggingFace framework