2 ms·
They really don't. Pretraining on low quality QA is a bad idea so at the very least they're filtering down the data set. Low quality QA is still useful for RL
by CuriouslyC 2y ago
They really don't. Pretraining on low quality QA is a bad idea so at the very least they're filtering down the data set. Low quality QA is still useful for RLHF/task training though if the question is well answered.
- sanxiyn 2y agoPeople often correct bad generation and I think such feedback is valuable. eg "Training Language Models with Language Feedback at Scale" https://arxiv.org/abs/2303.16755 https://arxiv.org/abs/2303.16755