2 ms·
What would the hardware requirement for fine-tuning a Llama 2 ?
by xmly 3y ago
What would the hardware requirement for fine-tuning a Llama 2 ?
- rushingcreek 3y agoWe didn't want to use LoRA to maximize quality, so we used 32 A100-80GB with a sequence length of 4096. It's possible to do a native fine-tune on as little as 8 A100-80GB with DeepSpeed Zero 3, but it will take longer. With LoRA you can probably get away with just a few 4090s.