4 ms·
Agreed—I hope Meta studied Deepseek's approach. The idea of a Deepseek Coder at 70B would be exciting.
by turnsout 3y ago
Agreed—I hope Meta studied Deepseek's approach. The idea of a Deepseek Coder at 70B would be exciting.
- whimsicalism 3y agoThe “approach” is likely just training on more tokens.
- imjonse 3y agoIt's the quality of data and the method of training, not just the amount of tokens (per their paper release a few days ago)
- hackerlight 3y agoThere's a deepseek coder around 30-35b and it has almost identical performance to the 7b on benchmarks.