2 ms·
You can customize a model with a "cheap" 3090 (or maybe a 7900 XTX), see here: https://github.com/Lightning-AI/lit-llama#finetune-the-model https://github.com/L
by brucethemoose2 4y ago
You can customize a model with a "cheap" 3090 (or maybe a 7900 XTX), see here: https://github.com/Lightning-AI/lit-llama#finetune-the-model https://github.com/Lightning-AI/lit-llama#finetune-the-model
What OP did is more intense, but Lora/Adapter still can give excellent results.
Pure AI cards for mere mortals aren't really a thing yet.
- theolivenbaum 4y agoAnyone tried fine tunning on CPU already? I expect it to be much slower, but is it even practical?
- brucethemoose2 4y agoOh no not a chance, finetuning is really compute intense.
- brucethemoose2 4y agoAlso I bet the pytorch training code is written with CUDA semantics. Maybe a JAX version would work without messing with the code.