7 ms·
Create chain-of-thought candidates with LLM, form preference pairs depending on whether the answers are correct, train using DPO and NLL, then repeat the cycle.
by Jimmc414 3y ago
Create chain-of-thought candidates with LLM, form preference pairs depending on whether the answers are correct, train using DPO and NLL, then repeat the cycle.
Apparently this takes Llama-2-70B-chat from 55.6% to 81.6% on the GSM8k benchmark
- careai 3y agothank you for your service
- popinman322 3y agoAlso, similar to Orca-Math but without a teacher model. They also followed an iterative DPO/KTO scheme, but with no length normalized NLL loss term.
- algo_trader 3y agoIf we had a magical (fast) oracle for grading responses, have people done search/expert iteration for LLMs? Specifically for codegen, i am playing with an iterative interpreter that can quickly (re)evaluate a tree of similar responses