3 ms·
I don't think they do, for a few reasons: 1. Evidence keeps adding up that the quality of your training data really matters. That's one of the reason's OpenAI
by simonw 2y ago
I don't think they do, for a few reasons:
1. Evidence keeps adding up that the quality of your training data really matters. That's one of the reason's OpenAI are signing big dollar deals with media companies The Atlantic, Vox etc have really high quality tokens!
2. Can you imagine the firestorm that will erupt the first time a company credibly claims that their private IP leaked to another user via a ChatGPT session that was later used as part of another response?
But as I said in the article... I can't say this with confidence because OpenAI are so opaque about what they actually use that data for! Which is really annoying.
- FeepingCreature 2y agoReally, what you'd want is to train a LoRA or something on user input. Then you can leave it specialized to that user.
- swiftcoder 2y ago> That's one of the reason's OpenAI are signing big dollar deals with media companies The Atlantic, Vox etc have really high quality tokens! They have really high quality tokens in their archive, at any rate. Since a bunch of media outlets have adopted GPT-powered writing tools, future tokens are presumably going to be far less valuable.
- lukan 2y ago"Since a bunch of media outlets have adopted GPT-powered writing tools, future tokens are presumably going to be far less valuable" As long as a human verified the output, I think it is fine. Training on unverified data is bad.
- swiftcoder 2y agoYou are still looping your own models output into the training set regardless. Human verification may avoid outright errors creeping in, but it won't stop the model biasing it's own training set