3 ms·

> We know RLHF and alignment degrades model quality. I feel you can't make statements like this without giving some sources. IIUC, without RLHF/alignment, the

• porridgeraisin 2y ago
  You're both right because RLHF and fine-tuning are just techniques. It's dependent on the training data and not as much the method. So, if you make the RLHF/finetune data such that it avoids certain topics, then you reduce model quality in practice since your training data might accidentally cast a net wide enough that you make it avoid certain legitimate questions. On benchmarks these things don't typically show up though. But yes. Those techniques are required for making it chat. Otherwise it just autocompletes from the internet. It is also used in a couple of other places (reasoning/search(hallucination mitigation))