17 ms·
If you play with a "raw" model such as LLaMA you'll find what you suggest is true. These models do what you'd expect of a model that was trained to predict the
by tel 3y ago
If you play with a "raw" model such as LLaMA you'll find what you suggest is true. These models do what you'd expect of a model that was trained to predict the next token.
It's quite tricky to convince such a model to do what you want. You have to conceptualize it and then imagine an optimal prefix leading to the sort of output you've conceptualized. That said, people discovered some fairly general-purpose prefixes, e.g.
Q: What is the 3rd law of Thermodynamics?
A:
This inspired the idea of "instruct tuning" of LLMs where fine-tuning techniques are applied to "raw" models to make them more amenable to completion of scripts where instructions are provided in a preamble and then examples of executions of those instructions follow.
This ends up being way more convenient. Now all the prompter has to do is conceptualize what they want and expect that the LLM will receive it as instruction. It simplifies prompting and makes the LLM more steerable, more useful, more helpful.
This is further refined through the use of explicit {:user}, {:assistant}, and {:system} tags which divide LLM contexts into different segments with explicit interpretations of the meaning of each segment. This is where "chat instruction" arises in models such as GPT-3.5.
- cosmojg 3y ago> It simplifies prompting and makes the LLM more steerable, more useful, more helpful. While this is true, there is also evidence that RLHF and supervised instruction tuning can hurt output quality and accuracy[1], which are instead better optimized through clever prompting[2]. [1] https://yaofu.notion.site/How-does-GPT-Obtain-its-Ability-Tracing-Emergent-Abilities-of-Language-Models-to-their-Sources-b9a57ac0fcf74f30a1ab9e3e36fa1dc1 https://yaofu.notion.site/How-does-GPT-Obtain-its-Ability-Tr... [2] https://yaofu.notion.site/Towards-Complex-Reasoning-the-Polaris-of-Large-Language-Models-c2b4a51355b44764975f88e6a42d4e75 https://yaofu.notion.site/Towards-Complex-Reasoning-the-Pola...
- jameshart 3y agoRight. But who's the 'you' who's being addressed by the {:system} prompt? Who is the {:assistant} supposed to think the {:system} is? Why should the {:assistant} output tokens that make it do what the {:system} tells it to? After all, the {:user} doesn't. The {:system} doesn't provide any instructions for how the {:user} is supposed to behave, the {:user} tokens are chosen arbitrarily and don't match the probabilities the model would have expected at all. This all just seems like an existential nightmare.
- noduerme 3y agoThe inculcation of the concepts of "you" and "assistant" into LLMs is definitely the start of a bad spiral.
- dingledork69 3y agoThat's just how the examples it's trained on are formatted
- astrange 3y agoWell, it just means we trained the model to work on instructions written that way. Since the result works out, that means the model must've learned to deal with it. There isn't much research on what's actually going on here, mainly because nobody has access to the weights of the really good models.
- mxkopy 3y agoIf I'm feeling romantic I think about a universal 'you' separate from the person that is referred to and is addressed by every usage of the word - a sort of ghost in the shell that exists in language. But really, it's probably just priming the responses to fit the grammatical structure of a first person conversation. That structure probably does a lot of heavy lifting in terms of how information is organized, too, so that's probably why you can see such qualitative differences when using these prompts.
- skissane 3y ago> If I'm feeling romantic I think about a universal 'you' separate from the person that is referred to and is addressed by every usage of the word - a sort of ghost in the shell that exists in language. That's not really romanticism, that's just standard English grammar – https://en.wikipedia.org/wiki/Generic_you https://en.wikipedia.org/wiki/Generic_you – it is the informal equivalent to the formal pronoun one. That Wikipedia article's claim that this is "fourth person" is not really standard. Some languages – the most famous examples are the Algonquian family – have two different third person pronouns, proximate (the more topically prominent third person) and obviative (the less topically prominent third person) – for example, if you were talking about your friend meeting a stranger, you might use proximate third person for your friend but obviative for the stranger. This avoids the inevitable clumsiness of English when describing interactions between two third persons of the same gender. Anyway, some sources describe the obviative third person as a "fourth person". And while English generic pronouns (generic you/one/he/they) are not an obviative third person, there is some overlap – in languages with the proximate-obviative distinction, the obviative often performs the function of generic pronouns, but it goes beyond that to perform other functions which purely generic pronouns cannot. You can see the logic of describing generic pronouns as "fourth person", but it is hardly standard terminology. I suspect this is a case of certain Wikipedia editors liking a phrase/term/concept and trying to use Wikipedia to promote/spread it.