4 ms·
You had the right understanding in your first comment, but what was missing was the fine tuning. You are right that there aren't many documents on the web that
by tinco 3y ago
You had the right understanding in your first comment, but what was missing was the fine tuning. You are right that there aren't many documents on the web that are structured that way, so the raw model wouldn't be very effective on predicting the next token.
But since we know that it will complete a command when structured it cleverly, all we had to do to fine tune it is synthesize (generate) a bazillion examples of documents that actually have the exact structure of a system or an assistant being told to do something, and then doing it.
Because it's seen many documents like that (that don't exist on the internet, only on the drives of OpenAI engineers) it knows how to predict the next token.
It's just a trick though, on top of the most magic thing which is that somewhere in those 175 billion weights or whatever it has, there is a model of the world that's so good that it could be easily fine tuned to understand this new context that it is in.
- Angostura 3y agoThank you - that was a splendidly clear explanation for something that also baffled me.
- jasongrout 3y agoAs an example, if you want to see what these sorts of things look like, Databricks open-sourced an instruction fine-tuning dataset sourced from their employees: https://huggingface.co/datasets/databricks/databricks-dolly-15k https://huggingface.co/datasets/databricks/databricks-dolly-... (disclaimer: I'm at Databricks)
- sdwr 3y agoThanks for the real life example! I don't like the bland, watered-down tone of ChatGPT, never put together that it's trained on unopinionated data. Feels like a tragedy of the commons thing, the average (or average publically acceptable) view of a group of people is bound to be boring.
- weinzierl 3y agoThank you for writing this up so clearly. A few pieces fell into place after reading your comment!
- tough 3y agoThat's a great way of explaining it.
- jameshart 3y agoYou’ve expressed this very well - Thank you. I get that the fine tuning is done over documents which are generated to encourage the dialog format. What I’m intrigued by is the way prompters choose to frame those documents. Because that is a choice. It’s a manufactured training set. Using the ‘you are an ai chatbot’ style of prompting, in all the samples we generate and give to the model, text attributed to {:system} is a voice of god who tells {:assistant} who to be; {:assistant} acts in accordance with {:system}’s instructions, and {:user} is a wildcard whose behavior is unrestricted. We’re training it by teaching it ‘there is a class of documents that transcribe the interactions between three entities, one of whom is obliged by its AI nature to follow the instructions of the system in order to serve the users’. I.e., sci-Fi stories about benign robot servants. And I wonder how much of the model’s ability to ‘predict how an obedient AI would respond’ is based on it having a broader model of how fictional computer intelligence is supposed to behave. We then use the resulting model to predict what the obedient ai would say next. Although hey - you could also use it to predict what the user will say next. But we prefer not to go there. But here’s the thing that bothers me: the approach of having {:system} tell {:assistant} who it will be and how it must behave rests not only on the prompt-writer anthropomorphizing the fictional ‘ai’ to tell it it’s nature - it relies on the LLM’s world model to then also anthropomorphize a fictional ai assistant that obeys those instructions, in order to predict what such a thing would say next if it existed. I don’t know why but I find this troubling. And part of what I find troubling is how casually people (prompters and users) are willing to go along with the ‘you are a chatbot’ fiction.
- frickinadorable 3y agoIt’s convenience, that’s all. If you think of a more convenient or effective way to prompt these models, that will be great! Your concerns sound to be of the “it’s problematic” category. Most such concerns are make believe outrage / pearl-clutching nonsense.
- jakelazaroff 3y agoYour comment would have been much better without the second paragraph.