One important but that is often left out is that ChatGPT is not the first model to come out using RLHF to train LLMs. As is typical in the AI field, Deepmind

One important but that is often left out is that ChatGPT is not the first model to come out using RLHF to train LLMs. As is typical in the AI field, Deepmind was key in the development of the process. Deepmind 's Sparrow came out just before ChatGPT (regarding language modeling with RLHF), and much of the RLHF work was explored in their robotics/agent exploration work just prior to application in language. OpenAI was integral in PPO, but it's important to know and understand it wasn't ChatGPT or OpenAI that is solely leading these advancements.