3 ms·
What do you need the extra 2 for? Tensor parallelism?
by Philpax 2mo ago
What do you need the extra 2 for? Tensor parallelism?
- arjie 2mo agoLonger context and more cache. The problem is that native format with DSpark enabled you have very little room on the VRAM.
- Philpax 2mo agoI was under the impression that you could fit the full 1M context within the 192GB VRAM as a result of DeepSeek's various architectural advancements, but I'll grant that DSpark + a larger pool for concurrency may necessitate more VRAM, yes.