3 ms·
This is simply a finetune of LLaMA-13B, so it has the same ~11GB RAM or VRAM requirement as 13B for inferencing. With 4bit LoRAs you can finetune 13B with 12GB
by MacsHeadroom 4y ago
This is simply a finetune of LLaMA-13B, so it has the same ~11GB RAM or VRAM requirement as 13B for inferencing.
With 4bit LoRAs you can finetune 13B with 12GB of VRAM. On a 3090 it can take as little as a couple of hours.