3 ms·
Fine-tune Llama 2 with DPO
- homarp 3y agoThis blog-post introduces the Direct Preference Optimization (DPO) method which is now available in the TRL library and shows how one can fine tune the recent Llama v2 7B-parameter model on the stack-exchange preference dataset which contains ranked answers to questions on the various stack-exchange portals.
- homarp 3y agoweights https://huggingface.co/kashif/stack-llama-2 https://huggingface.co/kashif/stack-llama-2