2 ms·
The author isn't entirely wrong here. However, his understanding of how human intervention is used in the system is wrong. ChatGPT uses GPT-3 as a baseline mode
by mammarahmed 4y ago
The author isn't entirely wrong here. However, his understanding of how human intervention is used in the system is wrong. ChatGPT uses GPT-3 as a baseline model. However, it finetunes it using a machine learning technique called Reinforcement Learning from Human Feedback (RLHF) This method uses human in the loop but not how the author thinks. The baseline model is finetuned on prompts and their answers (labels) provided by human labelers initially (which is a tedious task and doesn't scale well). Later this model is used to generate multiple answers for selected input prompt that human labelers rank based on the what they think is the most appropriate response. It's easier for labelers to rank responses compared to actually providing appropriate responses.
Even though the whole process is more complicated then what ive explained above, the model is essentially trained this way.
One of the reasons of releasing it for free for now is so they can gather data for further fine tuning the model (this is why after each answer there's a thumbs-up, thumps-down button for you to rate chatgpts responses)