3 ms·

The "why is DeepSeek so much cheaper than o1" question is currently a mystery, with the best guess being that compute in China is cheaper.

_lvbh 2y ago
The model is much smaller. Compute in China should be more expensive considering all the US restrictions
- minimaxir 2y ago
  No one knows the size of o1. The only hint was a paper that suggested it was 200B parameters. Meanwhile, DeepSeek R1 is known to be 671B parameters because it is open-source.
  - sigmoid10 2y ago
    R1 is a mixture model with only 37B active params. So while it's definitely expensive to train, it's rather light on compute during inference. All you really need is lots of memory.
- portaouflop 2y ago
  Electricity prices in China are about half of what they are in the US, I expect the rest is similar.
coliveira 2y ago
DeepSeek is use way less computing to achieve the same results. That's the whole difference.
- minimaxir 2y ago
  Then a US compute provider should be able to launch a similarly-priced competitor (e.g. to capture the enterprise market concerned about the China associations) using the open-source version and drastically undercut OpenAI. I suspect that won’t be the case.
  - rfoo 2y ago
    > Then a US compute provider should be able to launch a similarly-priced competitor Right, you just need a few months to implement efficient inference for MLA + their strangely looking MoE scheme + ..., easy! Oh wait, the inference scheme described in their tech report is pretty much an exact fit for H800s. So if you run the recipe on H100s you are wasting the potential of your H100s. Otherwise have fun making variations to the serving architecture. To be fair, we had chance. If someone decided to replicate the effort to serve their models back in May 2024 when DeepSeek-V2 was out we'd have it now. But nobody had interest as DS-V2 was pretty mediocre. They (and whoever realized the potential) made big bet and it is paying off.

_lvbh 2y ago

The model is much smaller. Compute in China should be more expensive considering all the US restrictions

minimaxir 2y ago
No one knows the size of o1. The only hint was a paper that suggested it was 200B parameters. Meanwhile, DeepSeek R1 is known to be 671B parameters because it is open-source.
- sigmoid10 2y ago
  R1 is a mixture model with only 37B active params. So while it's definitely expensive to train, it's rather light on compute during inference. All you really need is lots of memory.
portaouflop 2y ago
Electricity prices in China are about half of what they are in the US, I expect the rest is similar.

coliveira 2y ago

DeepSeek is use way less computing to achieve the same results. That's the whole difference.

minimaxir 2y ago
Then a US compute provider should be able to launch a similarly-priced competitor (e.g. to capture the enterprise market concerned about the China associations) using the open-source version and drastically undercut OpenAI. I suspect that won’t be the case.
- rfoo 2y ago
  > Then a US compute provider should be able to launch a similarly-priced competitor Right, you just need a few months to implement efficient inference for MLA + their strangely looking MoE scheme + ..., easy! Oh wait, the inference scheme described in their tech report is pretty much an exact fit for H800s. So if you run the recipe on H100s you are wasting the potential of your H100s. Otherwise have fun making variations to the serving architecture. To be fair, we had chance. If someone decided to replicate the effort to serve their models back in May 2024 when DeepSeek-V2 was out we'd have it now. But nobody had interest as DS-V2 was pretty mediocre. They (and whoever realized the potential) made big bet and it is paying off.