4 ms·

I am wondering when will harnesses generate their own RLHF/DPO training sets and just LoRA finetune the models they are running?