My understanding is that they fine tune the model. They fine tune it through prompt engineering (e.g everything that goes into chatgpt has a prompt attached) a

My understanding is that they fine tune the model. They fine tune it through prompt engineering (e.g everything that goes into chatgpt has a prompt attached) and they fine tune it through having hundreds of paid contractors chat with it. In deep learning, fine tuning usually refers to only training the top layers. That means that bill of training happens on gigantic corpora which teaches the model a very advanced feature extraction is the bottom and middle layers. Then the contractors retrain the top layers to make it behave more like it takes instructions