> One of the tricks OpenAI uses when fine-tuning models is to use the unadjusted foundation model as a coherency model alongside the data or reward model to be

> One of the tricks OpenAI uses when fine-tuning models is to use the unadjusted foundation model as a coherency model alongside the data or reward model to be fine-tuned on. Loss is calculated as the sum[0] of both the fine-tuning and coherency loss so that the training process is anchored to something. You mean the K-L constraint?