There's a lot of conflicting information here. There are two papers: https://arxiv.org/pdf/2412.19437v1 https://arxiv.org/pdf/2412.19437v1 for DeepSeek-V3-Base

There's a lot of conflicting information here. There are two papers: https://arxiv.org/pdf/2412.19437v1 https://arxiv.org/pdf/2412.19437v1 for DeepSeek-V3-Base, which makes claims about 2.8M H800 GPU hours, and https://arxiv.org/pdf/2501.12948 https://arxiv.org/pdf/2501.12948 for DeepSeek-R1, which was based on V3-Base but doesn't seem to quote the cost of its RL steps. https://newsletter.languagemodels.co/p/the-illustrated-deepseek-r1 https://newsletter.languagemodels.co/p/the-illustrated-deeps... is a really fascinating overview of what those new steps were! The innovation here seems to be that the creation of a cutting-edge reasoning model could be made from a relatively inexpensive base model, regardless of how much RL is needed. Part of this seems to be a novel methodology for generating large volumes of chain-of-thought training data; it's left ambiguous exactly how the bulk of this data was generated, though, and it could have been a long process of manual curation. I'd also take the assertion that they started R1 with V3-Base in its release form with a grain of salt. We'll see whether people can reproduce similar results. Either way, there's a new chain-of-thought model with open weights, and that's an incredible achievement regardless of the context - and one that will be meaningful to future research.