3 ms·
how much memory does the 7B training need?
by seydor 4y ago
how much memory does the 7B training need?
- eulers_secret 4y ago~7.5GB - it'll be the same as running inference with a full context. That's for 4-bit quantization, the 8-bit quantization uses more RAM than my 3080 has...
- seydor 4y agoI wonder how much it would take to train the 4b 13B
- MacsHeadroom 4y agoAbout 15GB training it in the webui. If you use https://github.com/johnsmith0031/alpaca_lora_4bit https://github.com/johnsmith0031/alpaca_lora_4bit then 30B only needs 24GB, and works on a single 3090 or $200 P40.