3 ms·
My question is: what kind of hardware do you need to run this Flash beast locally at a meaningful speed?
by Lucasoato 24d ago
My question is: what kind of hardware do you need to run this Flash beast locally at a meaningful speed?
- ekianjo 24d agoa beefy pc with at least 20 GPUs
- aenis 24d ago8x RTX PRO 6000 or 4x Spark? Or 1x M5 Ultra 512GB. The model is theoretically FP8, but really internally its mostly FP4 already, so there won't be a cut-in-half-but-almost-just-as-good quant coming for this one.
- segmondy 24d agoLots of GPU, be resourceful. Look for older GPUs and grab them when they are available. For less than the price of 1 Blackwell 6000 or Mac Studio 512gb, I can run these locally and much faster due to older GPUs I grabbed when there was deal to be found.