5 ms·

So I have a thought here… since ChatGPT is also trained with the content of websites wouldnt there be consequences if at some point it would be ingesting too mu

by tacker2000 4y ago

So I have a thought here… since ChatGPT is also trained with the content of websites wouldnt there be consequences if at some point it would be ingesting too much of its own output, or the output of other “AI’s”? Lets say in a couple of years the web and any other global content output gets to a share of 10-20% of “AI” generated content, that would definitely change what it spits out. The question is would that be bad or good? So in a sense it could be we are now at peak ChatGPT output quality, since the inputs given to it are 99% from humans.

seydor 4y ago
There must be an equilibrium point at some point. I assume the % of AI content will be far higher than 20% in a few years
nobu-mori 4y ago
This is definitely a valid concern. OpenAI did extensive data grooming to ensure high-quality inputs were used as training data. They went out of their way, for instance, to attempt to remove auto-translated content. One interesting thing is that the situation you describe provides a huge moat for the first successful system. For instance, OpenAI can store and fingerprint all of the output from GPT-3 and ChatGPT. They can use these fingerprints to prevent training newer versions of GPT on old outputs. Less popular systems won't be able to sanitize the training data as well.
nzoschke 4y ago
I read that GPT adds some “watermarking” to its content to help with detection of pure AI generated content. Certainly we already need tools for this just like those to detect plagiarism. Cat and mouse game I know but AIs that can detect AI content seems like a reasonable outcome to me.
- pixl97 4y ago
  Reasonable for a limited amount of time. Eventually down the line someone comes out with an adversarialy trained GPT model that causes a high enough false positive rate that humans trying to use the system get pissed off.
bilsbie 4y ago
Some would call that thinking…

seydor 4y ago

There must be an equilibrium point at some point. I assume the % of AI content will be far higher than 20% in a few years

nobu-mori 4y ago

This is definitely a valid concern. OpenAI did extensive data grooming to ensure high-quality inputs were used as training data. They went out of their way, for instance, to attempt to remove auto-translated content. One interesting thing is that the situation you describe provides a huge moat for the first successful system. For instance, OpenAI can store and fingerprint all of the output from GPT-3 and ChatGPT. They can use these fingerprints to prevent training newer versions of GPT on old outputs. Less popular systems won't be able to sanitize the training data as well.

nzoschke 4y ago

I read that GPT adds some “watermarking” to its content to help with detection of pure AI generated content. Certainly we already need tools for this just like those to detect plagiarism. Cat and mouse game I know but AIs that can detect AI content seems like a reasonable outcome to me.

pixl97 4y ago
Reasonable for a limited amount of time. Eventually down the line someone comes out with an adversarialy trained GPT model that causes a high enough false positive rate that humans trying to use the system get pissed off.

bilsbie 4y ago

Some would call that thinking…