3 ms·
Didn't know 24GB was considered low lol.
by idonotknowwhy 3y ago
Didn't know 24GB was considered low lol.
- mcbuilder 3y agoFor AI that's either a very fat SDXL model at it's max native resolution, or a quantized 34B parameter model, so it's on the low size. Compare that with the Blackwell AI "superchip" announced yesterday that appears to the programmer as single GPU with 30TB of RAM.
- michaelt 3y agoHere's a vram requirements table for fine-tuning an LLM: https://github.com/hiyouga/LLaMA-Factory?tab=readme-ov-file#hardware-requirement https://github.com/hiyouga/LLaMA-Factory?tab=readme-ov-file#... No matter how much vram you have, there's something that doesn't fit :)
- throwing_away 3y agoThis is also how I learned that 8X7B doesn't mean "eight 7B models joined somehow".