4 ms·
How does the model qualify as local? ~192 GB RAM needed sounds a bit much for local.
by aswegs8 5mo ago
How does the model qualify as local? ~192 GB RAM needed sounds a bit much for local.
- antirez 5mo agoRuns on 96GB MacBooks. 128GB is better. Check the README of DwarfStar.
- defen 5mo agoIf you were buying a computer today to use for DS4 (budget under 10k) what would you get? I care more about inference quality than speed. Or is it better to wait for the rumored M5 studio?
- bigyabai 5mo agoFor large contexts, you're probably going to want CUDA. The DGX Spark is expensive but would get you decent speeds on DS4 Flash. DS4 Pro is probably not possible with $10k if you want sub-10 minute prefill.
- embedding-shape 5mo agoCan you download it and run it given you have the hardware? Then it's local, regardless if you happen to have the needed hardware or not. Bit like asking if Zigbee can be considered local/LAN for people who don't have the required radio/antenna.