2 ms·
Can anyone explain the prompting format for this? <|prompter|>{prompt}</s><|assistant|> Why is that </s> in there? And how would I prompt it with a previ
by simonw 3y ago
Can anyone explain the prompting format for this?
<|prompter|>{prompt}</s><|assistant|>
Why is that </s> in there?
And how would I prompt it with a previous history of the conversation?
- simonw 3y agoI tried running a prompt through it just now: <|prompter|>hello</s><|assistant|> And got back this (truncated): hey there!</s> introducing the first of its kind, a revolutionary AI... Why did that </s> come back as part of the response?
- 65a 3y agoYou would want '</s>' as a stop token.
- brucethemoose2 3y ago</s> is the "end" token, its common in the instruct syntax for many LLMs. <|prompter|> is the human message (including the instructon/contexts from my tests) and <|assistant|> is the bot's response. There are llm wrappers that will save chat histories and reformat them appropriately for different LLMs. For instance, see TUI and its instruction template presets: https://github.com/oobabooga/text-generation-webui/tree/main/instruction-templates https://github.com/oobabooga/text-generation-webui/tree/main...
- simonw 3y agoAaah I see - so it's actually more like: <|prompter|>... prompt goes here</s> <|assistant|> ... assistant response will go here, maybe followed by </s> Still don't understand why I got a rogue </s> in the reply when I ran a prompt through it (but it continued after that tag): https://news.ycombinator.com/item?id=38101766 https://news.ycombinator.com/item?id=38101766 And is my guess about </s> at the end of the assistant thing correct? If so then a conversation might look like this: <|prompter|>hi</s> <|assistant|>Hi to you too. What do you want to talk about?</s> <|prompter|>Tell me facts about snakes</s> <|assistant|> But the documentation on these models is so thin! Do I need that </s> at the end of the assistant things or not?
- brucethemoose2 3y ago> But the documentation on these models is so thin! Welcome to LLM research land, lol. Mostly there is no documentation, no reproducability, just random hit-and-run releases with some claimed metrics and if you're lucky a model card with the trained prompt. > Do I need that </s> at the end of the assistant things or not? ...I dunno. I didn't add it in my prompt template, and you can see that the model itself it outputting that token in the responses. Maybe ask the trainers on the HF page.