6 ms·

It was evident that this will happen. > Compared with our initial baseline on the same hardware, we achieved a 3× improvement in end-to-end serving performance

It was evident that this will happen. > Compared with our initial baseline on the same hardware, we achieved a 3× improvement in end-to-end serving performance, reaching hardware efficiency and per-token cost comparable to mainstream NVIDIA GPUs. This demonstrates that Chinese chips can support frontier-model inference efficiently and economically at scale.

verdverm 15d ago
you can also derive some stats from the ~10T tokens a day on 100k devices, 100M / device / day, but then one has to account for the multi-gpu model size, and I need coffee before I go there
- a34729t 15d ago
  I mean all the database stuff is obvious low hanging fruit for inference engines.
gpt5 15d ago
I’ll just note that NVidia’s moat has never been inference, and there are many chips used at a much larger scale than Chinese chips for inference like TPUs and AMD chips. The other part is that it’s a bit of a meme here to say that the chip restriction is actually helping China (or shall I say, coordinated effort?). For once, we know that China has put a lot of pressure on the US to relax these controls multiple times. In addition to large chip smuggling networks (e.g. 22% of NVidia’s worldwide revenue magically comes from Singapore, and the ratio has been growing). Lastly, assuming acceleration in AI (which we ARE seeing), there might not be time to China to catch up. The best estimate right now is that the first EUV chips from China will not come out before 2030. By that time who knows how powerful AI will be. All I’m saying is that the discussion is so one sided and a bit baselesss with no nuance, that it seems either a meme/groupthink in the community or coordinated. If anything, the data suggests that the US should increase its export controls and better track the tech supply chain if it wants to further curb Chinese progress.
- miroljub 15d ago
  > All I’m saying is that the discussion is so one sided and a bit baselesss with no nuance, that it seems either a meme/groupthink in the community or coordinated. If anything, the data suggests that the US should increase its export controls and better track the tech supply chain if it wants to further curb Chinese progress. Or, to put it bluntly, you are a proponent of the trade war.
- edgyquant 15d ago
  The idea that China wasn’t working hard to build their own chips is a political talking point period. Nothing has been sped up here, China was always going to do this and the backlash you see from people who are against it is only proof that restrictions do hurt them
  - dominotw 15d ago
    > The idea that China wasn’t working hard to build their own chips is a political talking point period. never heard of this 'talking point' . who is even saying this?
    - stanfordkid 15d ago
      It's an implied corollary: when folks say that "blocking US Chips is what led China to build it's own" they are implying it is US export restrictions that pushed China to innovate. OP is saying this isn't necessarily true. I don't think it is either: China seems to have succeeded due to a centrally planned economy that values independence, with high amounts of strategic government led technology investment.
      - vcryan 15d ago
        It can be both. They were already doing it and the situation made it more urgent.
      - dominotw 15d ago
        > OP is saying this isn't necessarily true. but your supposed corollary was > China wasn’t working hard to build their own chips kind of dishonest ?
HSO 15d ago
[flagged]
- menaerus 15d ago
  Look into the history clases will be enough. Why would I need to prove anything to a random stranger on the internet?
- goodmythical 15d ago
  Isn't it common knowledge that every better mouse trap breeds smarter mice?
  - conmod278 15d ago
    Future AI systems will eke out every last blood drop of performance from any kind of hardware. Not even a single bit flip will go to waste.
- Tadpole9181 15d ago
  Third party here, I feel like I'm missing something. Your comment seems somewhat pointed and I don't understand what exactly you're asking for? Weren't they just saying that it's self-evident that preventing a manufacturing superpower - one with a significant pool of industrial engineers and effectively unlimited money & government backing - from acquiring some good is a temporary measure? Because if they have sufficient incentive, they would just... Learn to build it themselves? After all, their entire nation is built around building things, and catching up / leap-frogging is much easier than starting from scratch.
  - HSO 15d ago
    [flagged]
    - browningstreet 15d ago
      > in 2022, mainstream and westoid virtualism opinion was ``china is cut off from the future now`` (cue the clownface here). ``we cut off chinas legs`` yadayada maybe where you were paying attention. were they also certain that americans wouldn't be paying the tariffs? but lots of people who saw what china was doing with renewable energy and EVs rightly suggested this would put them on the path of independence w/r/t GPUs and LLMs. this isn't like the ol' days where "japan can't do software". we knew china can. your attitude isn't warranted.

verdverm 15d ago

you can also derive some stats from the ~10T tokens a day on 100k devices, 100M / device / day, but then one has to account for the multi-gpu model size, and I need coffee before I go there

a34729t 15d ago
I mean all the database stuff is obvious low hanging fruit for inference engines.

gpt5 15d ago

I’ll just note that NVidia’s moat has never been inference, and there are many chips used at a much larger scale than Chinese chips for inference like TPUs and AMD chips. The other part is that it’s a bit of a meme here to say that the chip restriction is actually helping China (or shall I say, coordinated effort?). For once, we know that China has put a lot of pressure on the US to relax these controls multiple times. In addition to large chip smuggling networks (e.g. 22% of NVidia’s worldwide revenue magically comes from Singapore, and the ratio has been growing). Lastly, assuming acceleration in AI (which we ARE seeing), there might not be time to China to catch up. The best estimate right now is that the first EUV chips from China will not come out before 2030. By that time who knows how powerful AI will be. All I’m saying is that the discussion is so one sided and a bit baselesss with no nuance, that it seems either a meme/groupthink in the community or coordinated. If anything, the data suggests that the US should increase its export controls and better track the tech supply chain if it wants to further curb Chinese progress.

miroljub 15d ago
> All I’m saying is that the discussion is so one sided and a bit baselesss with no nuance, that it seems either a meme/groupthink in the community or coordinated. If anything, the data suggests that the US should increase its export controls and better track the tech supply chain if it wants to further curb Chinese progress. Or, to put it bluntly, you are a proponent of the trade war.
edgyquant 15d ago
The idea that China wasn’t working hard to build their own chips is a political talking point period. Nothing has been sped up here, China was always going to do this and the backlash you see from people who are against it is only proof that restrictions do hurt them
- dominotw 15d ago
  > The idea that China wasn’t working hard to build their own chips is a political talking point period. never heard of this 'talking point' . who is even saying this?
  - stanfordkid 15d ago
    It's an implied corollary: when folks say that "blocking US Chips is what led China to build it's own" they are implying it is US export restrictions that pushed China to innovate. OP is saying this isn't necessarily true. I don't think it is either: China seems to have succeeded due to a centrally planned economy that values independence, with high amounts of strategic government led technology investment.
    - vcryan 15d ago
      It can be both. They were already doing it and the situation made it more urgent.
    - dominotw 15d ago
      > OP is saying this isn't necessarily true. but your supposed corollary was > China wasn’t working hard to build their own chips kind of dishonest ?

HSO 15d ago

[flagged]

menaerus 15d ago
Look into the history clases will be enough. Why would I need to prove anything to a random stranger on the internet?
goodmythical 15d ago
Isn't it common knowledge that every better mouse trap breeds smarter mice?
- conmod278 15d ago
  Future AI systems will eke out every last blood drop of performance from any kind of hardware. Not even a single bit flip will go to waste.
Tadpole9181 15d ago
Third party here, I feel like I'm missing something. Your comment seems somewhat pointed and I don't understand what exactly you're asking for? Weren't they just saying that it's self-evident that preventing a manufacturing superpower - one with a significant pool of industrial engineers and effectively unlimited money & government backing - from acquiring some good is a temporary measure? Because if they have sufficient incentive, they would just... Learn to build it themselves? After all, their entire nation is built around building things, and catching up / leap-frogging is much easier than starting from scratch.
- HSO 15d ago
  [flagged]
  - browningstreet 15d ago
    > in 2022, mainstream and westoid virtualism opinion was ``china is cut off from the future now`` (cue the clownface here). ``we cut off chinas legs`` yadayada maybe where you were paying attention. were they also certain that americans wouldn't be paying the tariffs? but lots of people who saw what china was doing with renewable energy and EVs rightly suggested this would put them on the path of independence w/r/t GPUs and LLMs. this isn't like the ol' days where "japan can't do software". we knew china can. your attitude isn't warranted.