3 ms·
I wish projects like these would mention about the minimum GPU VRAM one needs, for inference or fine-tuning.
by chompychop 3y ago
I wish projects like these would mention about the minimum GPU VRAM one needs, for inference or fine-tuning.
- MacsHeadroom 3y agoThis is simply a finetune of LLaMA-13B, so it has the same ~11GB RAM or VRAM requirement as 13B for inferencing. With 4bit LoRAs you can finetune 13B with 12GB of VRAM. On a 3090 it can take as little as a couple of hours.