2 ms·
It's also not helping the confusion that the distills themselves were made and released by DeepSeek. If you want the actual "lighter version" of the model the
by TeMPOraL 2y ago
It's also not helping the confusion that the distills themselves were made and released by DeepSeek.
If you want the actual "lighter version" of the model the usual way, i.e. third-party quants, there's a bunch of "dynamic quants" of the bona fide (non-distilled) R1 here: https://unsloth.ai/blog/deepseekr1-dynamic https://unsloth.ai/blog/deepseekr1-dynamic. The smallest of them is just able to barely run on a beefy desktop, at less than 1 token per second.