2 ms·
People are reproducing the training for Deepseek R1.
by kelipso 1y ago
People are reproducing the training for Deepseek R1.
- wkat4242 1y agoOh that's interesting, because they distilled only with 1 method, not both methods that were used on R1 (the one was reinforcement learning, the other supervised finetuning, I think only the former was used on the distillations). So there might be some room for improvement. I hadn't seen any third party distillations yet but I'll have a look.