4 ms·

Replying to a deleted comment, because I think this is worth underscoring: By explicitly supporting this, China could signal to AI researchers worldwide that C

Replying to a deleted comment, because I think this is worth underscoring: By explicitly supporting this, China could signal to AI researchers worldwide that China is a safe haven. It’s the materialization of what everyone said would happen: when copyright enthusiasts shut down AI work in America, companies will simply go elsewhere. Imagine if the epicenter of AI research shifted away from the US. This may seem unlikely, but lack of data is the biggest problem that open source LLMs face. Mistral had to completely refuse to disclose their training sources just to release a decent model. Personally, I’m tempted to focus on this full time. It’s what I’ve always wanted to do. I never gave a thought to legal consequences, and with this collection there appears to be none.

TerrifiedMouse 3y ago
> China is a safe haven Heh. It’s a safe haven until your LLM says something the CCP doesn’t like.
- FartyMcFarter 3y ago
  "Safe"and "haven" are indeed bold worlds when applied to the current Chinese regime...
- Vetch 3y ago
  This issue is overstated. Given the material is in Chinese, it stands to reason the bulk (but of course not all) of it will not be in violation of whatever policies. Furthermore, there have been a series of open-weights Chinese models that give reasonable answers to sensitive (in China) questions. Unless you plan to release something customer facing in China, it's not something to stress about. The idea that overly burdensome regulations around open source models in the US shifts the global center of mass of LLMs to China is not implausible. One reason resonant among government officials against opensource LLMs is to keep them from China and Russia. But among the very best embeddings, ~13Bs and perhaps 34B open-weights models are the Chinese ones. The recent DeepSeekCoder also tops EvalPlus: https://evalplus.github.io/leaderboard.html https://evalplus.github.io/leaderboard.html For those concerned about over-training, ChatGPT and GPT4 are almost certainly also contaminated and there have been independent confirmations of DeepSeekCoder's strength.
  - rfoo 3y ago
    > Given the material is in Chinese, it stands to reason the bulk (but of course not all) of it will not be in violation of whatever policies. Especially due to no freedom of the press in China (you need a permit for each published book) they are all already censored. I still don't think China could be a safe haven though, they haven't shot down open source models not because they are nice, it's just that they are incompetent.
    - TerrifiedMouse 3y ago
      > Especially due to no freedom of the press in China (you need a permit for each published book) they are all already censored. Even in a fully censored environment, you can still accidentally run afoul of state censors. https://www.theguardian.com/world/2022/jun/06/chinese-star-taken-offline-after-showing-tank-cake-on-tiananmen-anniversary https://www.theguardian.com/world/2022/jun/06/chinese-star-t...
  - corethree 3y ago
    No LLMs will evolve emergent behavior. It will have capabilities beyond the training data including saying shit about china. You just need the to prompt it correctly.
deleted 3y ago
[deleted]
stavros 3y ago
While this is true, imagine having to run your entire dataset past the censors, to make sure your LLM doesn't turn out to have unpalatable opinions.
- manquer 3y ago
  Only if the model serves the mainland. Chinese authorities don’t care what you serve others I think .
- greesil 3y ago
  In Soviet China, LLM censors you. Sort of joking, but some people seem to be using the dual LLM approach where the 2nd LLM censors the output of the first. Take the Gandalf game as an example. And while you don't need an LLM to censor an entire internet at scale I see how it would be a useful tool for the censors in China or anywhere else. "Hey CensorGPT does the following weibo post history make fun of dear leader, taking note of the themes of Winnie the Pooh, touch on any other criticism of the CPC" etc
- lucubratory 3y ago
  On a technical level it's really not that much harder than all of the RLHF companies have to do in the West.
  - stavros 3y ago
    I guess it wouldn't be, since it's for the exact same purpose. The west is just censoring different things than the east.
    - squigz 3y ago
      They both censor what they don't like
    - lucubratory 3y ago
      Well, there are some things different and some things the same. Every society censors some information, it is just part of state power. Examples of things that both Western nations and China censor are things like material support for their designated terrorist groups, fraudulent commercial communications, bomb making instructions, information tools that could be too helpful for producing bioweapons, the identities of personnel in intelligence, the identities of undercover police, the precise security arrangements for the head of state, etc. Also worth noting that censorship serves a dual purpose. Censorship can be bypassed both inside China and outside it, but because it's overwhelmingly administered through the internet the intelligence services will get a bunch of pings if you're trying to access information they care about. Censorship serves a secondary purpose of making you much more likely to generate intelligence if you go searching for XYZ restricted information. I am not familiar with how enforcement is done in China, but in the US it's common for the intelligence agencies to detect someone through search patterns (e.g. "what is the schedule of Joe Biden" "where can I buy a pressure cooker" "purchase micronised aluminium" will absolutely get you flagged somewhere if you google it and actually click on the links/engage normally), and then either go straight to entrapment if it can get through court, or engage in parallel construction if it can't (most common method is just convincing a cop's CI to give a tip about the suspect, which can trigger digital searches with a warrant) and then move to entrapment. I imagine in China they don't bother with parallel construction, there is no legal fiction that they're not surveilling you, so they can just convict you for their equivalent of conspiracy based on the digital evidence as presented to the Judge.
m3kw9 3y ago
Maybe they will need you to give them access to weights like how tech co need to share IPs with them.

TerrifiedMouse 3y ago

> China is a safe haven Heh. It’s a safe haven until your LLM says something the CCP doesn’t like.

FartyMcFarter 3y ago
"Safe"and "haven" are indeed bold worlds when applied to the current Chinese regime...
Vetch 3y ago
This issue is overstated. Given the material is in Chinese, it stands to reason the bulk (but of course not all) of it will not be in violation of whatever policies. Furthermore, there have been a series of open-weights Chinese models that give reasonable answers to sensitive (in China) questions. Unless you plan to release something customer facing in China, it's not something to stress about. The idea that overly burdensome regulations around open source models in the US shifts the global center of mass of LLMs to China is not implausible. One reason resonant among government officials against opensource LLMs is to keep them from China and Russia. But among the very best embeddings, ~13Bs and perhaps 34B open-weights models are the Chinese ones. The recent DeepSeekCoder also tops EvalPlus: https://evalplus.github.io/leaderboard.html https://evalplus.github.io/leaderboard.html For those concerned about over-training, ChatGPT and GPT4 are almost certainly also contaminated and there have been independent confirmations of DeepSeekCoder's strength.
- rfoo 3y ago
  > Given the material is in Chinese, it stands to reason the bulk (but of course not all) of it will not be in violation of whatever policies. Especially due to no freedom of the press in China (you need a permit for each published book) they are all already censored. I still don't think China could be a safe haven though, they haven't shot down open source models not because they are nice, it's just that they are incompetent.
  - TerrifiedMouse 3y ago
    > Especially due to no freedom of the press in China (you need a permit for each published book) they are all already censored. Even in a fully censored environment, you can still accidentally run afoul of state censors. https://www.theguardian.com/world/2022/jun/06/chinese-star-taken-offline-after-showing-tank-cake-on-tiananmen-anniversary https://www.theguardian.com/world/2022/jun/06/chinese-star-t...
- corethree 3y ago
  No LLMs will evolve emergent behavior. It will have capabilities beyond the training data including saying shit about china. You just need the to prompt it correctly.

deleted 3y ago

[deleted]

stavros 3y ago

While this is true, imagine having to run your entire dataset past the censors, to make sure your LLM doesn't turn out to have unpalatable opinions.

manquer 3y ago
Only if the model serves the mainland. Chinese authorities don’t care what you serve others I think .
greesil 3y ago
In Soviet China, LLM censors you. Sort of joking, but some people seem to be using the dual LLM approach where the 2nd LLM censors the output of the first. Take the Gandalf game as an example. And while you don't need an LLM to censor an entire internet at scale I see how it would be a useful tool for the censors in China or anywhere else. "Hey CensorGPT does the following weibo post history make fun of dear leader, taking note of the themes of Winnie the Pooh, touch on any other criticism of the CPC" etc
lucubratory 3y ago
On a technical level it's really not that much harder than all of the RLHF companies have to do in the West.
- stavros 3y ago
  I guess it wouldn't be, since it's for the exact same purpose. The west is just censoring different things than the east.
  - squigz 3y ago
    They both censor what they don't like
  - lucubratory 3y ago
    Well, there are some things different and some things the same. Every society censors some information, it is just part of state power. Examples of things that both Western nations and China censor are things like material support for their designated terrorist groups, fraudulent commercial communications, bomb making instructions, information tools that could be too helpful for producing bioweapons, the identities of personnel in intelligence, the identities of undercover police, the precise security arrangements for the head of state, etc. Also worth noting that censorship serves a dual purpose. Censorship can be bypassed both inside China and outside it, but because it's overwhelmingly administered through the internet the intelligence services will get a bunch of pings if you're trying to access information they care about. Censorship serves a secondary purpose of making you much more likely to generate intelligence if you go searching for XYZ restricted information. I am not familiar with how enforcement is done in China, but in the US it's common for the intelligence agencies to detect someone through search patterns (e.g. "what is the schedule of Joe Biden" "where can I buy a pressure cooker" "purchase micronised aluminium" will absolutely get you flagged somewhere if you google it and actually click on the links/engage normally), and then either go straight to entrapment if it can get through court, or engage in parallel construction if it can't (most common method is just convincing a cop's CI to give a tip about the suspect, which can trigger digital searches with a warrant) and then move to entrapment. I imagine in China they don't bother with parallel construction, there is no legal fiction that they're not surveilling you, so they can just convict you for their equivalent of conspiracy based on the digital evidence as presented to the Judge.

m3kw9 3y ago

Maybe they will need you to give them access to weights like how tech co need to share IPs with them.