4 ms·
deepseek v3 and r1 are both 700B models, who has that much memory to run the model locally these days?
by synergy20 2y ago
deepseek v3 and r1 are both 700B models, who has that much memory to run the model locally these days?
- z_ 2y agoExolabs claims they can distribute the compute over many machines to use memory in aggregate: https://github.com/exo-explore/exo https://github.com/exo-explore/exo Maybe there is enough memory in many machines.
- zamadatix 2y agoThat's the general task but the hard part is having the pile of local machines with nearly a TB of VRAM to distribute it on. You'd need over 30 3090s worth of GPUs to run those models.