3 ms·
We know RLHF and alignment degrades model quality. could it be that Grok, due to its less restrictive training guidelines (and the fact that its creators aren't
by behnamoh 2y ago
We know RLHF and alignment degrades model quality. could it be that Grok, due to its less restrictive training guidelines (and the fact that its creators aren't afraid of getting sued), can achieve higher performance partly due to this simple factor?
- nialv7 2y ago> We know RLHF and alignment degrades model quality. I feel you can't make statements like this without giving some sources. IIUC, without RLHF/alignment, the model won't even be able to chat with you, it would just be a document completion engine.
- porridgeraisin 2y agoYou're both right because RLHF and fine-tuning are just techniques. It's dependent on the training data and not as much the method. So, if you make the RLHF/finetune data such that it avoids certain topics, then you reduce model quality in practice since your training data might accidentally cast a net wide enough that you make it avoid certain legitimate questions. On benchmarks these things don't typically show up though. But yes. Those techniques are required for making it chat. Otherwise it just autocompletes from the internet. It is also used in a couple of other places (reasoning/search(hallucination mitigation))