4 ms·

> cheaper than the cost of local electricity only. Can you explain what that means?

simonw 7mo ago
I think they mean that the DeepSeek API charges are less than it would cost for the electricity to run a local model. Local model enthusiasts often assume that running locally is more energy efficient than running in a data center, but fail to take the economies of scale into account.
- littlestymaar 7mo ago
  I guess it mostly comes from using the model with batch-size = 1 locally, vs high batch size in a DC, since GPU consumption don't grow that much with batch size. Note that while a local chatbot user will mostly be using batch-size = 1, it's not going to be true if they are running an agentic framework, so the gap is going to narrow or even reverse.
  - eru 7mo ago
    Well, different parts of the world also have different electricity prices.
    - littlestymaar 7mo ago
      Usually not multiple orders of magnitude difference though.
- jacquesm 7mo ago
  Some of those local model enthusiasts can actually afford solar panels.
  - jLaForest 7mo ago
    You are still incurring a cost if you use the electricity instead of selling it back to the grid
    - Kodiack 7mo ago
      The extent of that heavily depends on where you are. Where I live in NZ, the grid export rates are very low while the import rates are very high. Our peak import rate is 3x higher than our solar export rate. In other words, we’d need to sell 3 kWh hours of energy to offset the cost of using 1 kWh at peak. We’re currently in the process of accepting a quote for home batteries. The rates here highly incentivise maximising self-use.
    - dmichulke 7mo ago
      Luxembourg: Purchase price = 2 x sales price, mostly due to grid costs. And this is with no income tax or VAT on sold electricity.
    - jacquesm 7mo ago
      Selling it back to the grid is something that is still possible but much, much less of a financially sound proposition than it was a few years ago because of regulatory capture by the utilities. In some places it is so bad that you get penalized for excess power. Local consumption is the fastest way to capitalize on this, more so if you can make money with that excess power.
- croes 7mo ago
  Local enthusiasts don’t have to fear account banning.
- BoredomIsFun 7mo ago
  > Local model enthusiasts often assume that running locally is more energy efficient than running in a data center, It is a well known 101 truism in /r/Localllama that local is rarely cheaper, unless run batched - then it is massively, 10x cheaper indeed. > I think they mean that the DeepSeek API charges are less than it would cost for the electricity to run a local model. Because it is hosted in China, where energy is cheap. In ex-USSR where I live it is inexpensive too, and keeping in mind that whole winter I had to use small space heater, due to inadequacy of my central heating, using local came out as 100% free.
- pbhjpbhj 7mo ago
  Is it economies of scale, or is it unpaid externalities?
jojobas 7mo ago
China has cheap electricity.
- ericd 7mo ago
  Well, also, LLM servers get much more efficient with request queue depth >1 - tokens per second per gpu are massively higher with 100 concurrents than 1 on eg vllm.
- DeathArrow 7mo ago
  Yes, but the hardware they use for inference like Huawei Ascend 910C is less efficient than Nvidia H100 used in US due to the difference in the process node.
atoav 7mo ago
It means that the electricity you would have to pay if you did the computations yourself would be more expensive than paying them to do it. Part of thst has to do with the fact that China has cheap electricity, also due to their massive push into renewables. Part of that is just economies of scale. A big server farm can run more efficiently than your PC on average.
- AuthAuth 7mo ago
  cheap electric due to their massive push on non renewables. There has been no change in the price of electricity during the renewable shift.
  - atoav 7mo ago
    Normslly you'd expext that more (and cheaper) supply would drive down prices. Classic market logic. How do do you explain that this market logic ceases to exist for renewables only? A whopping ~2TW or ~35% of generated power in China is renewable and since renewable energy is roughly 1.5 to 4 times cheaper than e.g. coal per kW/h produced that ought to have some impact. If it has not I'd be curious in your explaination of the mechanism involved.
    - AuthAuth 7mo ago
      China's power price has never reflected market costs. But the reason why you arent seeing it drop with so much renewables added is because your numbers are wrong. Its about 20-25% generated as renewables over the past 5 years and while a lot of renewable capacity and non renewable capacity has been added the power demand has raised to match. When we look at the current demand its a near inf scale whatever is available modern industry will consume. When we compare this to the power boom in the 2000s they were able to build enough energy generation to meet demand and were able to drive the price down from 30c rural and 17c city to 8cents for both.

I think they mean that the DeepSeek API charges are less than it would cost for the electricity to run a local model. Local model enthusiasts often assume that running locally is more energy efficient than running in a data center, but fail to take the economies of scale into account.

littlestymaar 7mo ago
I guess it mostly comes from using the model with batch-size = 1 locally, vs high batch size in a DC, since GPU consumption don't grow that much with batch size. Note that while a local chatbot user will mostly be using batch-size = 1, it's not going to be true if they are running an agentic framework, so the gap is going to narrow or even reverse.
- eru 7mo ago
  Well, different parts of the world also have different electricity prices.
  - littlestymaar 7mo ago
    Usually not multiple orders of magnitude difference though.
jacquesm 7mo ago
Some of those local model enthusiasts can actually afford solar panels.
- jLaForest 7mo ago
  You are still incurring a cost if you use the electricity instead of selling it back to the grid
  - Kodiack 7mo ago
    The extent of that heavily depends on where you are. Where I live in NZ, the grid export rates are very low while the import rates are very high. Our peak import rate is 3x higher than our solar export rate. In other words, we’d need to sell 3 kWh hours of energy to offset the cost of using 1 kWh at peak. We’re currently in the process of accepting a quote for home batteries. The rates here highly incentivise maximising self-use.
  - dmichulke 7mo ago
    Luxembourg: Purchase price = 2 x sales price, mostly due to grid costs. And this is with no income tax or VAT on sold electricity.
  - jacquesm 7mo ago
    Selling it back to the grid is something that is still possible but much, much less of a financially sound proposition than it was a few years ago because of regulatory capture by the utilities. In some places it is so bad that you get penalized for excess power. Local consumption is the fastest way to capitalize on this, more so if you can make money with that excess power.
croes 7mo ago
Local enthusiasts don’t have to fear account banning.
BoredomIsFun 7mo ago
> Local model enthusiasts often assume that running locally is more energy efficient than running in a data center, It is a well known 101 truism in /r/Localllama that local is rarely cheaper, unless run batched - then it is massively, 10x cheaper indeed. > I think they mean that the DeepSeek API charges are less than it would cost for the electricity to run a local model. Because it is hosted in China, where energy is cheap. In ex-USSR where I live it is inexpensive too, and keeping in mind that whole winter I had to use small space heater, due to inadequacy of my central heating, using local came out as 100% free.
pbhjpbhj 7mo ago
Is it economies of scale, or is it unpaid externalities?

jojobas 7mo ago

China has cheap electricity.

ericd 7mo ago
Well, also, LLM servers get much more efficient with request queue depth >1 - tokens per second per gpu are massively higher with 100 concurrents than 1 on eg vllm.
DeathArrow 7mo ago
Yes, but the hardware they use for inference like Huawei Ascend 910C is less efficient than Nvidia H100 used in US due to the difference in the process node.

atoav 7mo ago

It means that the electricity you would have to pay if you did the computations yourself would be more expensive than paying them to do it. Part of thst has to do with the fact that China has cheap electricity, also due to their massive push into renewables. Part of that is just economies of scale. A big server farm can run more efficiently than your PC on average.

AuthAuth 7mo ago
cheap electric due to their massive push on non renewables. There has been no change in the price of electricity during the renewable shift.
- atoav 7mo ago
  Normslly you'd expext that more (and cheaper) supply would drive down prices. Classic market logic. How do do you explain that this market logic ceases to exist for renewables only? A whopping ~2TW or ~35% of generated power in China is renewable and since renewable energy is roughly 1.5 to 4 times cheaper than e.g. coal per kW/h produced that ought to have some impact. If it has not I'd be curious in your explaination of the mechanism involved.
  - AuthAuth 7mo ago
    China's power price has never reflected market costs. But the reason why you arent seeing it drop with so much renewables added is because your numbers are wrong. Its about 20-25% generated as renewables over the past 5 years and while a lot of renewable capacity and non renewable capacity has been added the power demand has raised to match. When we look at the current demand its a near inf scale whatever is available modern industry will consume. When we compare this to the power boom in the 2000s they were able to build enough energy generation to meet demand and were able to drive the price down from 30c rural and 17c city to 8cents for both.