4 ms·
Chinese AI models have lagged the US frontier by 7 months on average since 2023
- jdmoreira 10mo agothat's because they are distilling the frontier models
- epolanski 10mo agoFrontier models aren't released, they are closed source. And the Chinese have been a huge source of innovation in the field.
- jdmoreira 10mo agodistilling does not require the models to be released. They simply use the apis. They have been a source of innovation but probably not in training them.
- NoOn3 10mo agoThey just lack of performant hardware. They have enough knowledge. And so they choose a more effective strategy without wasting resources on training from scratch.
- deleted 10mo ago[deleted]
- Tostino 10mo agoYou can do a rough distill through the APIs. You don't need the weights. It was much easier when companies had models on the /completion style APIs, because you could actually get the logits for each generation step, and use that as a dataset to fit your model to. That isn't to diminish the efforts of the Chinese developers though, they are great.
- riku_iki 10mo agois there any evidence that this is significant contribution? My intuition that one need ALOT api credits to distill such large models.
- anishgupta 10mo agoThey have been catching up pretty fast though. Since US have banned Nvidia chips export upto a certain extent to them they are coming up with more optimized training at least which is why US should be wary of them. China does more with less
- meisel 10mo agoHow much less? Their companies clearly have access to the latest GPUs.
- anishgupta 10mo agoThis video can bring some perspective https://www.youtube.com/watch?v=CrJJPlRO9bI https://www.youtube.com/watch?v=CrJJPlRO9bI around 5:11 They use H800 as opposed to major US ones on H100 (2-3x faster)
- eptcyka 10mo agoDefinitely. And there’s lots of wasteful training practices in Singapore too.
- jacquesm 10mo agoThis story has so many repeats by now it should be taught in schools. Constraints breed improvements, time and again. Whether in the petri dish or in a tech domain, if you want to see greatness substitute the latest and greatest for something that is two generations older and watch the miracle unfold. What doesn't kill you really does make you stronger.
- ilamont 10mo agoThe US and China regularly use export controls (like the Nvidia ban), joint ventures, and legislation as levers of power over the other side. For American and other non-PRC companies thinking of using Chinese models, doesn't this have to be balanced with the risk that the US or its leadership may kneecap the Chinese models through export controls, an executive order, or some other means?
- 1970-01-01 10mo ago
- dgemm 10mo agoState of the art 7 months ago is good enough for a lot of use cases.
- kingstnap 10mo agoI suppose this is a more forward looking post though. It's about whoever gets to whatever is awaiting us in the future 7 months before the other. And I guess the idea is is that there is this extreme inflection point in utility somewhere that makes it so getting there first gives you some incredible economic edge. It might not exist though. Like either utility plateaus and its bubble crash of the century time or it just keeps going up but without any specific point where you can differentiate something.
- kelipso 10mo agoFeels like that inflection point possibility passed a while ago since these models are starting to plateau in performance.
- marcosdumay 10mo agoThe GP is talking about recursive self-improvement. What yes, it's clear by now it's way beyond the capacity of those AIs, and the odds are pretty good it's impossible to a large extent (but some limited version of it may be possible).
- laterium 10mo agoWhat makes you think they plateaued?
- SoftTalker 10mo agoThat was my thought. At my work, 7 months can be less time than it takes to get a project organized around a high-level plan. It sounds like nothing.
- biophysboy 10mo agoExactly - for an average person's task, how different is AI today vs 7 months ago?
- kachapopopow 10mo agoso there has been quite a few instances of extensions logging AI chatbot conversations, I wonder if this is related to them training on that data given the accusations of deepseek 'stealing' chatgpt.
- sjm 10mo agoAnd how much have they spent compared to US companies? What of carbon footprint?
- warkdarrior 10mo agoChina generates twice as much clean energy compared to US, in raw gigawatt-hours: https://www.reuters.com/markets/commodities/chinas-power-grid-clean-up-puts-several-us-systems-shame-maguire-2025-03-07/#:~:text=CLEANING%20UP,-China's%20power%20system&text=That%20fossil%20fuel%20share%20compares,total%20electricity%20output%20by%2036%25.&text=Over%20the%20same%20period%2C%20U.S.,over%20the%20next%20five%20years. https://www.reuters.com/markets/commodities/chinas-power-gri...
- plagiarist 10mo agoLooks like they are not far off on fossil fuels as well. And at around 5x the population.
- nunodonato 10mo agoOn the flagships, maybe. But on open models (and especially the small ones), China is kicking ass.
- koakuma-chan 10mo agoAren't they just distills.
- mapontosevenths 10mo agoDoes it matter what they are if they outperform?
- koakuma-chan 10mo agoDo they outperform? In any case, they wouldn't exist if not for superior models they were distilled from.
- venndeezl 10mo agoFirst does not always equal superior. We "distilled" modern cars from Model-T. You still driving the car that was "first" off an assembly line? This is normal improvement to manufacture of stuff. Your handwavy "it was first so its winner-winner chicken dinner!" is little more than your personal form of expression. Since LLMs are a distill of web content, by your poorly defined metric, you must not value LLMs and AI companies? The content already existed! They're just a new indexing tool.
- 6510 10mo agoMust be strange to work hard in a competitive environment and have all customers pretend your product is always exactly the same. Each wizard school also seems to take a different approach and have different goals. Soon people will benchmark lawyers with Lego.
- 10mo ago
- dust42 10mo agoThey use the ECI - EpochAI Capability Index. Measured by the DCI the Chinese AI models are about 1.5 years ahead of US models. DCI = Dust42 Capability Index: MBP Max 64GB, Qwen3-80B MLX 4bit quant, 40 tokens per second. It is not on Claude Opus level but very, very useful if you have no internet, i.e. on a flight. And occasionally it surpasses even Opus by far and large. Opus is a pain in the neck once the coding task at hand surpasses its capabilities. Qwen3 is much better to guide to get step by step to a solution.
- wiseowise 10mo agoInteresting, does it mean they're optimizing for cases where China is disconnected from the internet completely?
- litbear2022 10mo agoLet's see who disconnected from the internet completely :)
- marcosdumay 10mo agoNo, there are datacenters in China they can still use if their internet is down.
- torginus 10mo agoAlmost all Chinese models are open weight research models. My theory is that these models serve the purpose of being relatively easy to run/tweak for researchers, and mainly serve to demonstrate the effectiveness of new techniques in training and inference, as well as the strength of AI labs that created them. They are not designed to be state of the art commercial models. By choosing bigger model sizes, running more training epochs, and drilling the models a bit more on benchmarking questions, I'm sure the Chinese could close the gap, but that would delay these models, make them more expensive and harder to run without showing any tangible research benefit. Also my 2c: I was perfectly happy with Sonnet 3.7 as of a year ago, if the Chinese have a model really as good as that (not only one that benchmarks as well), I'd definitely like to try it.
- zelphirkalt 10mo agoQuestion about English for natives: "[...] have lagged behind [..]" would be the grammatically correct version of the heading, I think. Or is "to lag" without "behind" actually a correct use? Is it merely headline-speak, news-speak, to make headlines shorter and convey more information in fewer words?
- SoftTalker 10mo agolag is a verb meaning "to fail to keep up" so IMO, lagging behind is redundant. It is not uncommon to hear, but strictly unnecessary.
- zdragnar 10mo agoLag as a verb does imply following behind, but it can also be a noun, such as in "a time lag". Adding behind after lag as a verb is more of a "because it sounds good", perhaps as a subconscious way to emphasize the verb, but it isn't a grammatical requirement at all. Leaving it off is almost certainly more to keep the headline short than anything else.
- Citizen8396 10mo agoIt is correct, if somewhat uncommon in daily use. I published an article awhile ago and the editor revised it similarly.
- elinear 10mo agoBoth versions are grammatically correct. "Lagged behind" is common for everyday speech, while using "lagged" as a direct verb is a standard, formal way to describe data gaps in business or news. So yes, the headline uses just "lagged" to save space.
- jbotz 10mo agoI'm not a native, but lived in the US for a quarter century. I think you're correct that that "lagged behind" is the correct version, but if you replaced "lagged" with "trailed" it would also be correct without the "behind". Language is very fluid and always evolving, so using "lagged" as one would previously have used "trailed" may soon be considered correct usage. Note also that these aren't really questions of grammar (syntax) but meaning (semantics). Does "lagged" mean the same thing as "trailed" in this kind of construction? It didn't some decades ago, but maybe it does today. Or will tomorrow.
- isoprophlex 10mo agoNice. So ~ half a year and we get open-weights Opus 4.5 capability? I'm all for that!
- hereme888 10mo agoImagine the size of those weights.
- isoprophlex 10mo agoSomeone else who's not a big, american vampire corp feeding on VC money can worry about that for me...
- llmslave 10mo agoSo basically the amount of time it takes for them to scrape existing US models and then train on that data. China doesnt have any intellectual property, they are just stealing
- petcat 10mo agonone of the American AI companies have IP either. They just scrape and steal everything too. They just started earlier.
- llmslave 10mo agousa companies invented these models
- petcat 10mo agoNot disputing that. But the vast amounts of data and information they were trained on that makes them useful was all stolen. It's thieves all the way down.
- lossolo 10mo agoChina produced foundational technologies (paper, compass, printing, gunpowder etc) long before the US existed, the US later built on global inventions too. Same here, LLM progress is cumulative and international, today's leaders won't be tomorrow's by default. All frontier US models are closed weight. It's great what Chinese are doing because open weights help everyone. Also there is a lot of research thanks to these open weights, look how much research is being done using Qwen models in US (Microsoft etc) and in the rest of the world.
- deleted 10mo ago[deleted]
- andy99 10mo agoI could not find GLM on there, it is one of the best open weight Chinese models now, probably not at the top but seems only a few months behind.
- Alifatisk 10mo agoCan't wait for GLM-5, it's currently being trained
- jacquesm 10mo ago7 months on average and decreasing. As the rate of progress in models slows down the head start of any entity in the field will slowly reduce until there is parity.