45 ms·

There is a hole in the boat's bottom due to Chinese models. They might not be as good but they are not bad either or at least I had hard time finding any issues

by wg0 5mo ago

There is a hole in the boat's bottom due to Chinese models. They might not be as good but they are not bad either or at least I had hard time finding any issues with Deepseekv4 Flash and Pro variants. They get their job done sometimes rarely giving up till they are done what they are after. So even for enterprise deployments, as the dust settles down, CFO/CTOs might find out that deploying on an internal cluster of GPUs is far more cheaper and reliable for their organisational needs than paying someone else for burned tokens.

ok123456 5mo ago
Qwen3.6:35b is good enough for a lot of stuff. I just used ollama with a shell script to tackle my directory of papers/literature. I converted the first 6 pages of each document to PNG, handed them off to Qwen, and told it to spit out BibTeX, including the abstract. Two days later it was done, and I didn't spend anything on "tokens."
- abyssin 5mo ago
  Why PNG? Isn’t an image format more expensive to process?
  - ok123456 5mo ago
    Not really. The model is good/fast at OCR, and preprocessing it actually makes it worse because academic paper formatting is very complicated. Sizes, positions, and equations are important.
    - culi 5mo ago
      what a strange world we live in where robots are WORSE at handling formatted stuff. I wonder what this means for the importance of semantic HTML to screenreaders
raincole 5mo ago
I had been saying this on HN repeatedly: people are going to use the smartest models for coding. They don't care how cheap your tokens are if they don't have the highest probability of solving your programming tasks. And I was dead wrong. Now I mostly use DeepSeek Pro myself.
- peheje 5mo ago
  I mean indsight is 20/20, but saying that is like saying "everyone will just use the best tools". That's not what we see most places in the world for most types of resources.
- dcchambers 5mo ago
  I think two things happened: 1. The sheer number of tokens that a coding agent can use flipped the math upside down on this equation. If you use the most expensive model for everything those costs quickly become untenable, even for software companies. 2. We realized many of the coding problems we're solving aren't incredibly difficult.
- simplyluke 5mo ago
  The other thing that's changing is more and more CFOs are looking at the AI spend in engineering departments and hitting the brakes. Token leaderboards were cool when the spend wasn't a double-digit-percent of the entire department's budget including salaries.
- weitendorf 5mo ago
  I pretty strongly feel the opposite way. Granted I have not used deepseek enough to “know” their model idiosyncrasies as well as Anthropic, so there is a partial skill issue. But I just find it really hard to justify using a less powerful model while I work. The most I’ve ever spent in a month extra on API tokens for my own work is $200, and I pay for the $200/mo Claude. I use these models quite a lot, though not idly (I usually just walk around and do other stuff until I know how im going to approach the next set of problems). So it costs me about $3000/year to get as much as I want of the best model available. Already that seems low enough to not be worth stressing out too much about optimizing it, because it feels like an indisputable good value, and trying to save money with a less powerful model would be optimizing for a $1000-$2000 saving at the expense of a large portion of my work taking longer or being more frustrating and iterative. That’s not a flex or anything, I get that in other countries $3000/yr is a lot of money for a software developer and also a lot of people would perhaps rationally be better off doing X% worse at work or spending Y% more time on tasks to save $Z, if their productivity improvements didn’t translate to more salary. Otherwise if your performance has more upside I really do think that the smartest models are better with the current pricing scheme. Deepseek and the other Chinese models spend a LOT of time thinking, and tend to be much more jagged (benchmaxxed) in performance. How can dealing with that over an entire year be worth $2k? The only situation I can think of where sacrificing my own time/performance to save on inference is batch compute (of course, $1k vs $100k is different from $30 vs $3k) or work where the tier 2 models have crossed the “good enough” threshold. But I think Opus is not even close to that threshold generally yet. As it gets smarter I, and I think most others probably, just try to do harder things faster and hit the next wall.
pants2 5mo ago
The Chinese models are only cheap on subsidized Chinese hosting. I have yet to find a USA-hosted Chinese model with a very clear value advantage over US models.
- __mharrison__ 5mo ago
  Odd take. I'm running them locally at my desk (DGX Spark and 128GB MBP). They work fine for 90% of what most folks do. Admittedly, they do run slower on my hw than on the cloud.
  - pants2 5mo ago
    Running them locally is cool and has privacy/autonomy benefits, but you can't really make a value case for it. Guaranteed if you run the math you will never run enough inference to pay off your hardware vs buying tokens. Last time I ran the math on my MBP I'd have to run inference 24 hours a day for 5+ years to pay off the cost of my MBP, not accounting for electricity costs.
    - iooi 5mo ago
      Is this because of the tok/s? Since it's pretty easy to run up a $5k bill in API usage for Claude/ChatGPT in a month.
      - pants2 5mo ago
        Yes, because of the limits on tok/s, and you have to compare apples to apples, not Gemma 27B to Opus 4.7.
        hedora 5mo ago
        Assuming the local models get the job done (e.g., you adjust your workflow so that you can run the local machine 100% all the time, or whatever), then the time to payback isn't very high. MSRP for a 128GB AMD was $1400 at launch. That's 7 months of claude code subscription. If you assume a 5 year depreciation cycle, you can buy a cluster of 8 such machines and still come out ahead. (Power is a few hundred watts per machine peak -- maybe 7 machines if you include electricity.) Of course, I'm assuming non-bubble numbers. Those boxes are like $3K now. Still, a normal person would probably not buy 8 of them at once. Instead, they'd space out buying a machine every few years as the technology improves. For me, things are getting better faster than my ability to review / trust the resulting code, so tok/sec isn't a bottleneck anymore. Instead, quality of the tokens is the bottleneck. That points to me wanting a 1TB DRAM iGPU once they're available at pre-bubble RAM pricing.
SoftTalker 5mo ago
> CFO/CTOs might find out that deploying on an internal cluster of GPUs is far more cheaper and reliable I think you're right especially if you're someplace that already has a data center, such as a university. Solves a lot of privacy concerns as well.
surgical_fire 5mo ago
I am having some great experience with DeepSeek. In fact, it seems to perform better than Claude or Codex in my use case. I don't see myself returning to Claude or Codex anytime soon.
mariopt 5mo ago
I’ve been using Kimi 2.6, GLM 5.1 , Minimax 2.7 and lately deepseek. I only spend 40$ a month and I don’t see the point in paying for Opus/Codex. Chinese models are really quite good at a lot of stuff.
- fittingopposite 5mo ago
  Which harness?
  - mariopt 5mo ago
    I use opencode with all of them except Kimi, I noticed Kimi performs better with kimi-cli and also save a bit of quota. Z.ai does recommend to use claude cli as a harness for GLM5.1, I still get good results with opencode.
ihsw 5mo ago
[dead]
reppap 5mo ago
The problem with going for open source models is that you are betting on some third party to keep doing expensive model training and releasing it for free, forever. What do you do if deepseek never release another update to the model?
- julianlam 5mo ago
  I continue to use the model I downloaded... for free?
replwoacause 5mo ago
Anybody know what the most capable Chinese model is that can be used in production and is cheaper than US frontier models? Would that still be Deepseek? My interest is getting as close to Gpt5.5 or Opus quality as I can get, but for less $.
- culi 5mo ago
  Depends what you want it for. Probably Qwen https://arena.ai/leaderboard https://arena.ai/leaderboard
raylad 5mo ago
Possibly a deliberate strategy by the Chinese to undermine the US AI industry, data centers, and basically everything that’s powering the economy. Just like they did with the US steel industry in the 80s.

by wg0 5mo ago

ok123456 5mo ago

Qwen3.6:35b is good enough for a lot of stuff. I just used ollama with a shell script to tackle my directory of papers/literature. I converted the first 6 pages of each document to PNG, handed them off to Qwen, and told it to spit out BibTeX, including the abstract. Two days later it was done, and I didn't spend anything on "tokens."

abyssin 5mo ago
Why PNG? Isn’t an image format more expensive to process?
- ok123456 5mo ago
  Not really. The model is good/fast at OCR, and preprocessing it actually makes it worse because academic paper formatting is very complicated. Sizes, positions, and equations are important.
  - culi 5mo ago
    what a strange world we live in where robots are WORSE at handling formatted stuff. I wonder what this means for the importance of semantic HTML to screenreaders

raincole 5mo ago

I had been saying this on HN repeatedly: people are going to use the smartest models for coding. They don't care how cheap your tokens are if they don't have the highest probability of solving your programming tasks. And I was dead wrong. Now I mostly use DeepSeek Pro myself.

peheje 5mo ago
I mean indsight is 20/20, but saying that is like saying "everyone will just use the best tools". That's not what we see most places in the world for most types of resources.
dcchambers 5mo ago
I think two things happened: 1. The sheer number of tokens that a coding agent can use flipped the math upside down on this equation. If you use the most expensive model for everything those costs quickly become untenable, even for software companies. 2. We realized many of the coding problems we're solving aren't incredibly difficult.
simplyluke 5mo ago
The other thing that's changing is more and more CFOs are looking at the AI spend in engineering departments and hitting the brakes. Token leaderboards were cool when the spend wasn't a double-digit-percent of the entire department's budget including salaries.
weitendorf 5mo ago
I pretty strongly feel the opposite way. Granted I have not used deepseek enough to “know” their model idiosyncrasies as well as Anthropic, so there is a partial skill issue. But I just find it really hard to justify using a less powerful model while I work. The most I’ve ever spent in a month extra on API tokens for my own work is $200, and I pay for the $200/mo Claude. I use these models quite a lot, though not idly (I usually just walk around and do other stuff until I know how im going to approach the next set of problems). So it costs me about $3000/year to get as much as I want of the best model available. Already that seems low enough to not be worth stressing out too much about optimizing it, because it feels like an indisputable good value, and trying to save money with a less powerful model would be optimizing for a $1000-$2000 saving at the expense of a large portion of my work taking longer or being more frustrating and iterative. That’s not a flex or anything, I get that in other countries $3000/yr is a lot of money for a software developer and also a lot of people would perhaps rationally be better off doing X% worse at work or spending Y% more time on tasks to save $Z, if their productivity improvements didn’t translate to more salary. Otherwise if your performance has more upside I really do think that the smartest models are better with the current pricing scheme. Deepseek and the other Chinese models spend a LOT of time thinking, and tend to be much more jagged (benchmaxxed) in performance. How can dealing with that over an entire year be worth $2k? The only situation I can think of where sacrificing my own time/performance to save on inference is batch compute (of course, $1k vs $100k is different from $30 vs $3k) or work where the tier 2 models have crossed the “good enough” threshold. But I think Opus is not even close to that threshold generally yet. As it gets smarter I, and I think most others probably, just try to do harder things faster and hit the next wall.

pants2 5mo ago

The Chinese models are only cheap on subsidized Chinese hosting. I have yet to find a USA-hosted Chinese model with a very clear value advantage over US models.

__mharrison__ 5mo ago
Odd take. I'm running them locally at my desk (DGX Spark and 128GB MBP). They work fine for 90% of what most folks do. Admittedly, they do run slower on my hw than on the cloud.
- pants2 5mo ago
  Running them locally is cool and has privacy/autonomy benefits, but you can't really make a value case for it. Guaranteed if you run the math you will never run enough inference to pay off your hardware vs buying tokens. Last time I ran the math on my MBP I'd have to run inference 24 hours a day for 5+ years to pay off the cost of my MBP, not accounting for electricity costs.
  - iooi 5mo ago
    Is this because of the tok/s? Since it's pretty easy to run up a $5k bill in API usage for Claude/ChatGPT in a month.
    - pants2 5mo ago
      Yes, because of the limits on tok/s, and you have to compare apples to apples, not Gemma 27B to Opus 4.7.
      - hedora 5mo ago
        Assuming the local models get the job done (e.g., you adjust your workflow so that you can run the local machine 100% all the time, or whatever), then the time to payback isn't very high. MSRP for a 128GB AMD was $1400 at launch. That's 7 months of claude code subscription. If you assume a 5 year depreciation cycle, you can buy a cluster of 8 such machines and still come out ahead. (Power is a few hundred watts per machine peak -- maybe 7 machines if you include electricity.) Of course, I'm assuming non-bubble numbers. Those boxes are like $3K now. Still, a normal person would probably not buy 8 of them at once. Instead, they'd space out buying a machine every few years as the technology improves. For me, things are getting better faster than my ability to review / trust the resulting code, so tok/sec isn't a bottleneck anymore. Instead, quality of the tokens is the bottleneck. That points to me wanting a 1TB DRAM iGPU once they're available at pre-bubble RAM pricing.

SoftTalker 5mo ago

> CFO/CTOs might find out that deploying on an internal cluster of GPUs is far more cheaper and reliable I think you're right especially if you're someplace that already has a data center, such as a university. Solves a lot of privacy concerns as well.

surgical_fire 5mo ago

I am having some great experience with DeepSeek. In fact, it seems to perform better than Claude or Codex in my use case. I don't see myself returning to Claude or Codex anytime soon.

mariopt 5mo ago

I’ve been using Kimi 2.6, GLM 5.1 , Minimax 2.7 and lately deepseek. I only spend 40$ a month and I don’t see the point in paying for Opus/Codex. Chinese models are really quite good at a lot of stuff.

fittingopposite 5mo ago
Which harness?
- mariopt 5mo ago
  I use opencode with all of them except Kimi, I noticed Kimi performs better with kimi-cli and also save a bit of quota. Z.ai does recommend to use claude cli as a harness for GLM5.1, I still get good results with opencode.

ihsw 5mo ago

[dead]

reppap 5mo ago

The problem with going for open source models is that you are betting on some third party to keep doing expensive model training and releasing it for free, forever. What do you do if deepseek never release another update to the model?

julianlam 5mo ago
I continue to use the model I downloaded... for free?

replwoacause 5mo ago

Anybody know what the most capable Chinese model is that can be used in production and is cheaper than US frontier models? Would that still be Deepseek? My interest is getting as close to Gpt5.5 or Opus quality as I can get, but for less $.

culi 5mo ago
Depends what you want it for. Probably Qwen https://arena.ai/leaderboard https://arena.ai/leaderboard

raylad 5mo ago

Possibly a deliberate strategy by the Chinese to undermine the US AI industry, data centers, and basically everything that’s powering the economy. Just like they did with the US steel industry in the 80s.