4 ms·
>>>"You could for example DPO based on another LLM's evalution of how a conversation is going," You can update the weights based on a single point of data (res
by staticman2 3y ago
>>>"You could for example DPO based on another LLM's evalution of how a conversation is going,"
You can update the weights based on a single point of data (response was bad) but you probably can't usefully update a model that way.
- candiodari 3y agoI can't seem to find anyone actually usefully trying this. Do you know of any data?
- staticman2 3y agoNo but Google (for example) recommends fine-tuning based on at least 500 examples of question response pairs. DPO as far as I know requires a good and bad example. It's not a technique based on just saying "ai response is bad".