2 ms·
Those new Tesla T4s are looking cheap for 8GB VRAM. Are they useful for training a model or are they too far out of date, and is it more effective to use a sing
by joegibbs 3y ago
Those new Tesla T4s are looking cheap for 8GB VRAM. Are they useful for training a model or are they too far out of date, and is it more effective to use a single 4070 or similar?
- michaelt 3y agoDepends what model you want to train, and how well you want your computer to keep working while you're doing it. If you're interested in large language models there's a table of vram requirements for fine-tuning at [1] which says you could do the most basic type of fine-tuning on a 7B parameter model with 8GB VRAM. You'll find that training takes quite a long time, and as a lot of the GPU power is going on training, your computer's responsiveness will suffer - even basic things like scrolling in your web browser or changing tabs uses the GPU, after all. Spend a bit more and you'll probably have a better time. [1] https://github.com/hiyouga/LLaMA-Factory?tab=readme-ov-file#hardware-requirement https://github.com/hiyouga/LLaMA-Factory?tab=readme-ov-file#...