4 ms·
Not true. There's plenty of easy ways to update the weights if you want to do that. You could for example DPO based on another LLM's evalution of how a convers
by candiodari 3y ago
Not true. There's plenty of easy ways to update the weights if you want to do that.
You could for example DPO based on another LLM's evalution of how a conversation is going, or you could use whether the conversation keeps going.
- staticman2 3y ago>>>"You could for example DPO based on another LLM's evalution of how a conversation is going," You can update the weights based on a single point of data (response was bad) but you probably can't usefully update a model that way.
- candiodari 3y agoI can't seem to find anyone actually usefully trying this. Do you know of any data?
- staticman2 3y agoNo but Google (for example) recommends fine-tuning based on at least 500 examples of question response pairs. DPO as far as I know requires a good and bad example. It's not a technique based on just saying "ai response is bad".