2 ms·
How do you normally do DPO? Is that built in to PyTorch or something? Theoretically the hard part is collecting the examples with rejections etc.
by ilaksh 3y ago
How do you normally do DPO? Is that built in to PyTorch or something?
Theoretically the hard part is collecting the examples with rejections etc.
- patelajay285 3y agoCollecting data is hard, but the library is also a synthetic data generation library, so for example you can create the data for DPO fully synthetically, check out the self-rewarding LLMs example: https://datadreamer.dev/docs/latest/pages/get_started/quick_ https://datadreamer.dev/docs/latest/pages/get_started/quick_...