3 ms·
Flash next only needs 64GB for core model inference. If you really wanted it. Need 64GB vram, 900+ GB/s speed, and a lot of system ram (128GB). Seems feasible.
by bitexploder 11d ago
Flash next only needs 64GB for core model inference. If you really wanted it.
Need 64GB vram, 900+ GB/s speed, and a lot of system ram (128GB). Seems feasible. Hmm.
Maybe older GPUs work.
- hedgehog 11d ago128GB on a Ryzen 395 is enough but not by a lot. Around 90GB for weights plus 10GB to 20GB for KV cache and checkpoints. I don't know about 64GB of VRAM for less than around $2500 by itself.
- bitexploder 11d agoTesla V100? Would have to be a 3-ish bit quant in 64GB ram