3 ms·
With that amount of memory can you run 4-bit DeepSeek 4 Flash? It is way more efficient in the KV cache department so may be worth a try
by undefuser 4mo ago
With that amount of memory can you run 4-bit DeepSeek 4 Flash? It is way more efficient in the KV cache department so may be worth a try
- CamperBob2 4mo agoI haven't looked into DS4 yet but based on antirez's results on 128 GB Macbooks, it shouldn't be a problem to run it on a pair of RTX6000 Pros. Also see https://www.reddit.com/r/LocalLLaMA/comments/1sv649s/to_run_deepseek_v4_flash_how_much_max_vram_we/ https://www.reddit.com/r/LocalLLaMA/comments/1sv649s/to_run_... .