6 ms·

pretty great. How long before deepseek charges for each api call?

minimaxir 2y ago
DeepSeek does charge for the API (just 20-30x cheaper than o1), I assume OP is eating the cost. https://api-docs.deepseek.com/quick_start/pricing/ https://api-docs.deepseek.com/quick_start/pricing/
- andrewfromx 2y ago
  i see and where is https://groq.com/ https://groq.com/ in this. They used to be the cheapest?
  - minimaxir 2y ago
    The "why is DeepSeek so much cheaper than o1" question is currently a mystery, with the best guess being that compute in China is cheaper.
    - _lvbh 2y ago
      The model is much smaller. Compute in China should be more expensive considering all the US restrictions
      - minimaxir 2y ago
        No one knows the size of o1. The only hint was a paper that suggested it was 200B parameters. Meanwhile, DeepSeek R1 is known to be 671B parameters because it is open-source.
        sigmoid10 2y ago
        R1 is a mixture model with only 37B active params. So while it's definitely expensive to train, it's rather light on compute during inference. All you really need is lots of memory.
      - portaouflop 2y ago
        Electricity prices in China are about half of what they are in the US, I expect the rest is similar.
    - coliveira 2y ago
      DeepSeek is use way less computing to achieve the same results. That's the whole difference.
      - minimaxir 2y ago
        Then a US compute provider should be able to launch a similarly-priced competitor (e.g. to capture the enterprise market concerned about the China associations) using the open-source version and drastically undercut OpenAI. I suspect that won’t be the case.
        rfoo 2y ago
        > Then a US compute provider should be able to launch a similarly-priced competitor Right, you just need a few months to implement efficient inference for MLA + their strangely looking MoE scheme + ..., easy! Oh wait, the inference scheme described in their tech report is pretty much an exact fit for H800s. So if you run the recipe on H100s you are wasting the potential of your H100s. Otherwise have fun making variations to the serving architecture. To be fair, we had chance. If someone decided to replicate the effort to serve their models back in May 2024 when DeepSeek-V2 was out we'd have it now. But nobody had interest as DS-V2 was pretty mediocre. They (and whoever realized the potential) made big bet and it is paying off.
  - KTibow 2y ago
    Right now Deepseek's official hosting is cheaper than everyone else who can manage to run the model, including Deepinfra. I haven't seen any good hypotheses as to why other than their large batch size and speculative decoding.
    - rfoo 2y ago
      DeepSeek-V2/V3/R1's model architecture is very different from what Fireworks/Together/... were used to. That's their "business" model (okay, they don't care about business that much for now, but still) too: you can't run it efficiently without doing months of work we already did, so even with all weights open you can't compete with us.
  - minimaxir 2y ago
    Speaking of which, Groq just announced the distilled Llama 70B variant: https://x.com/GroqInc/status/1883734298488180826 https://x.com/GroqInc/status/1883734298488180826
quantadev 2y ago
[flagged]
- andrewfromx 2y ago
  wow I just asked deep "is taiwan independent from china?" and then the same question to claude. Very different.
- tokioyoyo 2y ago
  So, ChatGPT/Gemini's free version is stealing the users from European AI companies, and the goal is to harm them? Or we call that just a competition?
  - gpm 2y ago
    I think there's a legitimate argument to be made that ChatGPT and Gemini's free versions constitute unfair (and potentially even illegal) predatory pricing. They aren't even charging the price of the inference compute, which has got to make it much much harder for competing businesses to survive. I.e. https://en.wikipedia.org/wiki/Predatory_pricing https://en.wikipedia.org/wiki/Predatory_pricing / https://www.ftc.gov/advice-guidance/competition-guidance/guide-antitrust-laws/single-firm-conduct/predatory-or-below-cost-pricing https://www.ftc.gov/advice-guidance/competition-guidance/gui...
  - quantadev 2y ago
    That's a false equivalence. There's more nuance than that. Both OpenAI and Google have a genuine private-sector business model, whereas my claim is that DeepSeek is totally a China Gov't entity.
    - jazzyjackson 2y ago
      Why is it an interesting distinction whether a product is privately funded vs tax payer funded? To me it sounds like sour grapes.
      - 2y ago

DeepSeek does charge for the API (just 20-30x cheaper than o1), I assume OP is eating the cost. https://api-docs.deepseek.com/quick_start/pricing/ https://api-docs.deepseek.com/quick_start/pricing/

andrewfromx 2y ago
i see and where is https://groq.com/ https://groq.com/ in this. They used to be the cheapest?
- minimaxir 2y ago
  The "why is DeepSeek so much cheaper than o1" question is currently a mystery, with the best guess being that compute in China is cheaper.
  - _lvbh 2y ago
    The model is much smaller. Compute in China should be more expensive considering all the US restrictions
    - minimaxir 2y ago
      No one knows the size of o1. The only hint was a paper that suggested it was 200B parameters. Meanwhile, DeepSeek R1 is known to be 671B parameters because it is open-source.
      - sigmoid10 2y ago
        R1 is a mixture model with only 37B active params. So while it's definitely expensive to train, it's rather light on compute during inference. All you really need is lots of memory.
    - portaouflop 2y ago
      Electricity prices in China are about half of what they are in the US, I expect the rest is similar.
  - coliveira 2y ago
    DeepSeek is use way less computing to achieve the same results. That's the whole difference.
    - minimaxir 2y ago
      Then a US compute provider should be able to launch a similarly-priced competitor (e.g. to capture the enterprise market concerned about the China associations) using the open-source version and drastically undercut OpenAI. I suspect that won’t be the case.
      - rfoo 2y ago
        > Then a US compute provider should be able to launch a similarly-priced competitor Right, you just need a few months to implement efficient inference for MLA + their strangely looking MoE scheme + ..., easy! Oh wait, the inference scheme described in their tech report is pretty much an exact fit for H800s. So if you run the recipe on H100s you are wasting the potential of your H100s. Otherwise have fun making variations to the serving architecture. To be fair, we had chance. If someone decided to replicate the effort to serve their models back in May 2024 when DeepSeek-V2 was out we'd have it now. But nobody had interest as DS-V2 was pretty mediocre. They (and whoever realized the potential) made big bet and it is paying off.
- KTibow 2y ago
  Right now Deepseek's official hosting is cheaper than everyone else who can manage to run the model, including Deepinfra. I haven't seen any good hypotheses as to why other than their large batch size and speculative decoding.
  - rfoo 2y ago
    DeepSeek-V2/V3/R1's model architecture is very different from what Fireworks/Together/... were used to. That's their "business" model (okay, they don't care about business that much for now, but still) too: you can't run it efficiently without doing months of work we already did, so even with all weights open you can't compete with us.
- minimaxir 2y ago
  Speaking of which, Groq just announced the distilled Llama 70B variant: https://x.com/GroqInc/status/1883734298488180826 https://x.com/GroqInc/status/1883734298488180826

quantadev 2y ago

[flagged]

andrewfromx 2y ago
wow I just asked deep "is taiwan independent from china?" and then the same question to claude. Very different.
tokioyoyo 2y ago
So, ChatGPT/Gemini's free version is stealing the users from European AI companies, and the goal is to harm them? Or we call that just a competition?
- gpm 2y ago
  I think there's a legitimate argument to be made that ChatGPT and Gemini's free versions constitute unfair (and potentially even illegal) predatory pricing. They aren't even charging the price of the inference compute, which has got to make it much much harder for competing businesses to survive. I.e. https://en.wikipedia.org/wiki/Predatory_pricing https://en.wikipedia.org/wiki/Predatory_pricing / https://www.ftc.gov/advice-guidance/competition-guidance/guide-antitrust-laws/single-firm-conduct/predatory-or-below-cost-pricing https://www.ftc.gov/advice-guidance/competition-guidance/gui...
- quantadev 2y ago
  That's a false equivalence. There's more nuance than that. Both OpenAI and Google have a genuine private-sector business model, whereas my claim is that DeepSeek is totally a China Gov't entity.
  - jazzyjackson 2y ago
    Why is it an interesting distinction whether a product is privately funded vs tax payer funded? To me it sounds like sour grapes.
    - 2y ago