4 ms·

This post reminded me that I’m still somewhat surprised ChatGPT-like systems did not arise out of China first due to their massive investment funds for AI and t

This post reminded me that I’m still somewhat surprised ChatGPT-like systems did not arise out of China first due to their massive investment funds for AI and their base of stem talent. Once they catch-up their censors will have a super power and people will have their writings censored before they hit submit. Will be an interesting decade.

FooBarWidget 4y ago
This is already possible using simple Bayesian filters.
prepend 4y ago
> somewhat surprised ChatGPT-like systems did not arise out of China first due to their massive investment funds for AI and their base of stem talent. Does China innovate in this way? I think software is not a linear function of money in, value out. Inventing new programs and techniques is interesting as I’m sure what the spark is, and it certainly takes money, but that’s not the most important part. I remember reading in Masters of Doom about how John Carmack came up with some of the innovations id software used and was amazed that he, alone, came up with stuff that large, well-funded teams could not. I expect that China could have invested its entire GDP into stem and AI for the past 10 years and would not have came up with LLMs and ChatGPT. Or perhaps they did, but kept it secret and would never release it as a product to the public. But I believe the former based on the few software companies and few software innovations coming out of China in the past 20 years.
shanebellone 4y ago
IMO China is a "long-term country". LLM does not lead to General Intelligence. While the former is profitable (financially and socially), the latter is the holy grail. Time will name the winner.
- chaxor 4y ago
  It heavily depends upon the definition of general intelligence, because we haven't really settled on a definition as of yet. In some circles (of course the ones from Microsoft) the list of requirements is very close to being met by some of the more sophisticated multimodal models (GPT-4 et al). General intelligence is a strange question because it's more about the data/environment than the architecture, (obviously if it were about theoretical capabilities of architecture, AGI was found ages ago). Much of the LLM capabilities have to do with the generality baked into language. Pairing that with the well known proof that GNNs are dynamic programmers, you can get some very nice properties directly from the setup.
rerx 4y ago
US export controls certainly must slow down China somewhat. They can't import Nvidia's fastest GPUs, but are stuck with buying hobbled variations. Getting a domestic industry on par is hard.
tiedieconderoga 4y ago
The censors are one of the main reasons why China is so far behind on AI R&D. No unfiltered training data, and severe penalties if the AI hallucinates something that is against the party line.

FooBarWidget 4y ago

This is already possible using simple Bayesian filters.

prepend 4y ago

> somewhat surprised ChatGPT-like systems did not arise out of China first due to their massive investment funds for AI and their base of stem talent. Does China innovate in this way? I think software is not a linear function of money in, value out. Inventing new programs and techniques is interesting as I’m sure what the spark is, and it certainly takes money, but that’s not the most important part. I remember reading in Masters of Doom about how John Carmack came up with some of the innovations id software used and was amazed that he, alone, came up with stuff that large, well-funded teams could not. I expect that China could have invested its entire GDP into stem and AI for the past 10 years and would not have came up with LLMs and ChatGPT. Or perhaps they did, but kept it secret and would never release it as a product to the public. But I believe the former based on the few software companies and few software innovations coming out of China in the past 20 years.

shanebellone 4y ago

IMO China is a "long-term country". LLM does not lead to General Intelligence. While the former is profitable (financially and socially), the latter is the holy grail. Time will name the winner.

chaxor 4y ago
It heavily depends upon the definition of general intelligence, because we haven't really settled on a definition as of yet. In some circles (of course the ones from Microsoft) the list of requirements is very close to being met by some of the more sophisticated multimodal models (GPT-4 et al). General intelligence is a strange question because it's more about the data/environment than the architecture, (obviously if it were about theoretical capabilities of architecture, AGI was found ages ago). Much of the LLM capabilities have to do with the generality baked into language. Pairing that with the well known proof that GNNs are dynamic programmers, you can get some very nice properties directly from the setup.

rerx 4y ago

US export controls certainly must slow down China somewhat. They can't import Nvidia's fastest GPUs, but are stuck with buying hobbled variations. Getting a domestic industry on par is hard.

tiedieconderoga 4y ago

The censors are one of the main reasons why China is so far behind on AI R&D. No unfiltered training data, and severe penalties if the AI hallucinates something that is against the party line.