8 ms·

How does data removal from GPT *work

OpenAI has a form where you can request that your data is “removed” from their models https://share.hsforms.com/1UPy6xqxZSEqTrGDh4ywo_g4sk30 , but how does that actually work? How does one untrain a model? Are they planning to just retrain the entire model without your data? My understanding of LLMs is mostly theoretical so maybe I’m missing something here but my understanding is that once a model is trained you can’t exactly untrain it. It’s not like you can just move the weights a gradient step in the opposite direction or something. Idk what OpenAI plans to do with data removal requests and how effective it will actually be.

nickthegreek 3y ago
This would only be on future training.
maebert 3y ago
It's about as hard to remove a single data point from an LLM as it is to remove a single memory from a human brain. OpenAI doesn't remove data from the trained models, they filter it at the output level. They also remove it from training data, but of course the model lives on. I'm a strong supporter of a "do not encode" header / metadata on content that allows individuals, content creators and providers to tell AIs not to encode specific text / images / documents in the first place.
- Oras 3y ago
  Filtering would be extremely slow especially when streaming the data.
  - tss_caterpillar 3y ago
    Would filtration be slow? ChatGPT already seems to have some filtration on the output level due to the fact that it returns one of its template responses when it’s queried something regarded as harmful. Although I imagine filtering out a list of specific information might be less straight forward than that.
    - cuteboy19 3y ago
      It does it after the stream is already complete, the output is sent to some other model to know whether it's offensive but it's possible to simply view the raw output if you just cancel the generation midway
- xg15 3y ago
  How would "filter at the output level" even work? The output is arbitrary unstructured text. I don't see how that output could be reliably filtered except by another LLM.
- albert_e 3y ago
  > It's about as hard to remove a single data point from an LLM as it is to remove a single memory from a human brain. Sounds like there is a decent story that can be written about the right-to-forget in GPT/LLM era ... similar to "The Eternal Sunshine of the Spotless Mind" for human memory. "Hallucinations" will be a main story-telling device in this one as well.
kamray23 3y ago
A very interesting thing to do would be to prove that an LLM model could output your personal details, then request that as an EU citizen you would like to invoke your right to be forgotten. It'd be very interesting to see what kind of interpretability and re-training research could come out of attempting to avoid situations in which you ossify data into the model, specifically due to wanting to avoid legal repercussions.
- throwaway313313 3y ago
  Consider the AI Sec Ops future in the EU where in the run up to an election a political operative makes all widely used AIs "forget" about the competition by submitting a flurry of fraudulent right to be forgotten requests. This is no stretch of the imagination due to existence of fraudulent copyright takedown requests being submitted daily to Youtube against legitimate channels (with Youtube giving said legitimate channels very real copyright strikes against their accounts regularly, with only very visible cases having a chance of reversal)...
bhickey 3y ago
Transformer editing is an active field of research. https://arxiv.org/abs/2202.05262 https://arxiv.org/abs/2202.05262 https://arxiv.org/abs/2210.07229 https://arxiv.org/abs/2210.07229
- cuteboy19 3y ago
  Doesn't this harm the claim that the models do not contain any information about a specific image by an artist?
  - lxgr 3y ago
    If so, maybe that says something about that claim?
  - fortyseven 3y ago
    I think there's probably a big difference between how data is stored in art models versus how LLMs store textual information.
tommek4077 3y ago
This is like blurring houses in Google Street view in Germany. The house is still there.
hgsgm 3y ago
If (misnomer) OpenAI can't explain it to you, the removal is BS.
ozten 3y ago
This video [1] explains how LoRA and Hypernetworks work. This is one way to change the output of the model without retraining the entire model. 9:55 LoRA 14:45 Hypernetworks [1] https://www.youtube.com/watch?v=dVjMiJsuR5o https://www.youtube.com/watch?v=dVjMiJsuR5o
dantraztrev 3y ago
Not entirely sure but here is an hunch , a single memory is really hard to remove, and presuming they're not training it again. Here are some solutions 1) Block Personally Identifiable info to get into training data : Maybe have a redact such info before passing it to model . Just having a layer between I/O and the processing . 2) Have a sort of VCS for models and rewind and retrain if such is needed to be done . ( Seems quite unfeasible ) 3) If it is not an info but rather an opinion , train the model to counter biases until a netural or agreed stance is reached
kristianp 3y ago
Are there examples of gpt mentioning people like that? Can you craft a specific prompt to get it to hallucinate info about someone non-famous without mentioning them in the prompt?
segmondy 3y ago
I don't think they can. I believe they just add filtering in the data pipeline before & after the generation. Think about how they currently filter out "bad things" user input ---> | prompt filter | GPT | output filter | --> sanitized output
deleted 3y ago
[deleted]

nickthegreek 3y ago

This would only be on future training.

maebert 3y ago

It's about as hard to remove a single data point from an LLM as it is to remove a single memory from a human brain. OpenAI doesn't remove data from the trained models, they filter it at the output level. They also remove it from training data, but of course the model lives on. I'm a strong supporter of a "do not encode" header / metadata on content that allows individuals, content creators and providers to tell AIs not to encode specific text / images / documents in the first place.

Oras 3y ago
Filtering would be extremely slow especially when streaming the data.
- tss_caterpillar 3y ago
  Would filtration be slow? ChatGPT already seems to have some filtration on the output level due to the fact that it returns one of its template responses when it’s queried something regarded as harmful. Although I imagine filtering out a list of specific information might be less straight forward than that.
  - cuteboy19 3y ago
    It does it after the stream is already complete, the output is sent to some other model to know whether it's offensive but it's possible to simply view the raw output if you just cancel the generation midway
xg15 3y ago
How would "filter at the output level" even work? The output is arbitrary unstructured text. I don't see how that output could be reliably filtered except by another LLM.
albert_e 3y ago
> It's about as hard to remove a single data point from an LLM as it is to remove a single memory from a human brain. Sounds like there is a decent story that can be written about the right-to-forget in GPT/LLM era ... similar to "The Eternal Sunshine of the Spotless Mind" for human memory. "Hallucinations" will be a main story-telling device in this one as well.

kamray23 3y ago

A very interesting thing to do would be to prove that an LLM model could output your personal details, then request that as an EU citizen you would like to invoke your right to be forgotten. It'd be very interesting to see what kind of interpretability and re-training research could come out of attempting to avoid situations in which you ossify data into the model, specifically due to wanting to avoid legal repercussions.

throwaway313313 3y ago
Consider the AI Sec Ops future in the EU where in the run up to an election a political operative makes all widely used AIs "forget" about the competition by submitting a flurry of fraudulent right to be forgotten requests. This is no stretch of the imagination due to existence of fraudulent copyright takedown requests being submitted daily to Youtube against legitimate channels (with Youtube giving said legitimate channels very real copyright strikes against their accounts regularly, with only very visible cases having a chance of reversal)...

bhickey 3y ago

Transformer editing is an active field of research. https://arxiv.org/abs/2202.05262 https://arxiv.org/abs/2202.05262 https://arxiv.org/abs/2210.07229 https://arxiv.org/abs/2210.07229

cuteboy19 3y ago
Doesn't this harm the claim that the models do not contain any information about a specific image by an artist?
- lxgr 3y ago
  If so, maybe that says something about that claim?
- fortyseven 3y ago
  I think there's probably a big difference between how data is stored in art models versus how LLMs store textual information.

tommek4077 3y ago

This is like blurring houses in Google Street view in Germany. The house is still there.

hgsgm 3y ago

If (misnomer) OpenAI can't explain it to you, the removal is BS.

ozten 3y ago

This video [1] explains how LoRA and Hypernetworks work. This is one way to change the output of the model without retraining the entire model. 9:55 LoRA 14:45 Hypernetworks [1] https://www.youtube.com/watch?v=dVjMiJsuR5o https://www.youtube.com/watch?v=dVjMiJsuR5o

dantraztrev 3y ago

Not entirely sure but here is an hunch , a single memory is really hard to remove, and presuming they're not training it again. Here are some solutions 1) Block Personally Identifiable info to get into training data : Maybe have a redact such info before passing it to model . Just having a layer between I/O and the processing . 2) Have a sort of VCS for models and rewind and retrain if such is needed to be done . ( Seems quite unfeasible ) 3) If it is not an info but rather an opinion , train the model to counter biases until a netural or agreed stance is reached

kristianp 3y ago

Are there examples of gpt mentioning people like that? Can you craft a specific prompt to get it to hallucinate info about someone non-famous without mentioning them in the prompt?

segmondy 3y ago

I don't think they can. I believe they just add filtering in the data pipeline before & after the generation. Think about how they currently filter out "bad things" user input ---> | prompt filter | GPT | output filter | --> sanitized output

deleted 3y ago

[deleted]