4 ms·

I think they dumb down their public models to be only slightly better than the competition. And the real competition is China, so the current state of the Chine

by ozgung 2mo ago

I think they dumb down their public models to be only slightly better than the competition. And the real competition is China, so the current state of the Chinese models would define the baseline. I think one evidence is that the US has more than 5x the compute of China. With that difference in training speed, it should be impossible for Chinese models to close the gap that easily. It's also very unlikely that they sell the same public models to their private customers (military etc). We also know they talk about "unpublished internal models" for things like the last HuggingFace hacking incident. So it's not a bad theory. https://epoch.ai/publications/trends-in-ai-supercomputers https://epoch.ai/publications/trends-in-ai-supercomputers

piloto_ciego 2mo ago
This is kind of my intuition as well. I suspect that the models we don’t see are decidedly better than the models we do see.
- deleted 2mo ago
  [deleted]
embedding-shape 2mo ago
> I think one evidence is that the US has more than 5x the compute of China. With that difference in training speed, it should be impossible How could we really know how much "compute China has" in reality? Is it possible that whatever estimates people has come up with for both China and the US might not be 100% accurate?
- dibujaron 2mo ago
  I'm not an expert but I think this sort of thing is relatively traceable for two reasons. One, datacenters are difficult to conceal. Two, the supply chains for many of the relevant materials are difficult to conceal. Some of those supply chains still require western components, I believe, so if you know how much of X component was sent to china, you know how much compute they have.
- natmaka 2mo ago
  In China and in the US most owners of computing power have to quickly gain from it, as obsolescence hits hard. In China a consensual will emitted by powerful companies may convince the central power to subsidize efforts towards int'l market domination: R&D, including dataset building, learning... Maybe even also low prices obtained by selling at a price inferior to the costs...
usef- 2mo ago
There have been many reports that they're training in other countries. (On mobile so can't search, but this was yesterday:) > "Oracle was providing a staggering 22.6 percent of China's known A.I. computing power" In Malaysia, etc. https://thezvi.substack.com/p/ai-180-no-longer-in-charge https://thezvi.substack.com/p/ai-180-no-longer-in-charge
bjackman 2mo ago
> With that difference in training speed, it should be impossible for Chinese models to close the gap that easily. They have a big advantage in that they can directly distill from frontier models.

by ozgung 2mo ago

piloto_ciego 2mo ago

This is kind of my intuition as well. I suspect that the models we don’t see are decidedly better than the models we do see.

deleted 2mo ago
[deleted]

embedding-shape 2mo ago

> I think one evidence is that the US has more than 5x the compute of China. With that difference in training speed, it should be impossible How could we really know how much "compute China has" in reality? Is it possible that whatever estimates people has come up with for both China and the US might not be 100% accurate?

dibujaron 2mo ago
I'm not an expert but I think this sort of thing is relatively traceable for two reasons. One, datacenters are difficult to conceal. Two, the supply chains for many of the relevant materials are difficult to conceal. Some of those supply chains still require western components, I believe, so if you know how much of X component was sent to china, you know how much compute they have.
natmaka 2mo ago
In China and in the US most owners of computing power have to quickly gain from it, as obsolescence hits hard. In China a consensual will emitted by powerful companies may convince the central power to subsidize efforts towards int'l market domination: R&D, including dataset building, learning... Maybe even also low prices obtained by selling at a price inferior to the costs...

usef- 2mo ago

There have been many reports that they're training in other countries. (On mobile so can't search, but this was yesterday:) > "Oracle was providing a staggering 22.6 percent of China's known A.I. computing power" In Malaysia, etc. https://thezvi.substack.com/p/ai-180-no-longer-in-charge https://thezvi.substack.com/p/ai-180-no-longer-in-charge

bjackman 2mo ago

> With that difference in training speed, it should be impossible for Chinese models to close the gap that easily. They have a big advantage in that they can directly distill from frontier models.