Have you had a chance to look at the Direct Preference Optimization [1] pre-print? It seems like it might help get around RLHF which (as far as I can tell) is

Have you had a chance to look at the Direct Preference Optimization [1] pre-print? It seems like it might help get around RLHF which (as far as I can tell) is the really hard/expensive part of training the best of these models. They said they will release the code soon, so I guess we will found out soon enough. [1] https://arxiv.org/abs/2305.18290 https://arxiv.org/abs/2305.18290