3 ms·

Don't transformers typically have a <bot> token at the beginning of the prompt? This seems equivalent to letting the network attend to this token, and produce a

Don't transformers typically have a <bot> token at the beginning of the prompt? This seems equivalent to letting the network attend to this token, and produce a zero value if that's what it wants.

sp332 3y ago
Chat-tuned ones do, but the base models don't. For example, Llama doesn't, but Alpaca has "### Instruction:", "### Input:", and "### Response:".
- int_19h 3y ago
  Base LLaMA still has dedicated tokens for beginning/end of string. What you're describing is the instruction format, which is separate.
  - sp332 3y ago
    Oh, I had misunderstood something.
refulgentis 3y ago
not a token, and not the transformers, but yes, commercial chat models are fine-tuned on text transcripts containing dialogues. (i believe llama-2 was as well)
- sebzim4500 3y ago
  Are you sure? I have never seen an LLM that did not have a special token for start of text, I'm certain that llama had one and I don't remember anywhere in the llama-2 paper where they said they removed it.
  - refulgentis 3y ago
    tl;dr: you're right it's messy though, bear with me for the full explanation: - your initial post says "<bot>" token, which looked like a mix of "chatbot" and ChatML, used by OpenAI - there is a bo_S_ token, which acts as you described - I averaged my attention over your post and the initial reply, which answers as if you were using "<bot>" in the misunderstood way - when I go back and read your post, I realize the chatbot interpretation doesn't quite make sense, since you're referring to much more technical aspects than general "how do I AI", i.e. you understand <X> as a way to denote special tokens, not necessarily an XML tag
a1k0n 3y ago
Yes, it has to in fact. If you have zero context to attend to in a transformer, and you try to predict the first token, you effectively are multiplying a zero-vector by the attention head, making all tokens equally likely in the final softmax (unless the lm_head has a bias, but at least in GPT it does not). So the <|beginning of text|> token, with no context before it, learns to predict the first-token-in-a-document distribution. That's not quite the same as predicting nothing at all.
llwu 3y ago
This is a cool idea - it could be tested it by seeing whether W_V tends to map <bot> to 0, and whether <bot> tends to get attended to. There are interesting things to said whether or not this turns out to be the case.

Don't transformers typically have a <bot> token at the beginning of the prompt? This seems equivalent to letting the network attend to this token, and produce a zero value if that's what it wants.

sp332 3y ago

Chat-tuned ones do, but the base models don't. For example, Llama doesn't, but Alpaca has "### Instruction:", "### Input:", and "### Response:".

int_19h 3y ago
Base LLaMA still has dedicated tokens for beginning/end of string. What you're describing is the instruction format, which is separate.
- sp332 3y ago
  Oh, I had misunderstood something.

refulgentis 3y ago

not a token, and not the transformers, but yes, commercial chat models are fine-tuned on text transcripts containing dialogues. (i believe llama-2 was as well)

sebzim4500 3y ago
Are you sure? I have never seen an LLM that did not have a special token for start of text, I'm certain that llama had one and I don't remember anywhere in the llama-2 paper where they said they removed it.
- refulgentis 3y ago
  tl;dr: you're right it's messy though, bear with me for the full explanation: - your initial post says "<bot>" token, which looked like a mix of "chatbot" and ChatML, used by OpenAI - there is a bo_S_ token, which acts as you described - I averaged my attention over your post and the initial reply, which answers as if you were using "<bot>" in the misunderstood way - when I go back and read your post, I realize the chatbot interpretation doesn't quite make sense, since you're referring to much more technical aspects than general "how do I AI", i.e. you understand <X> as a way to denote special tokens, not necessarily an XML tag

a1k0n 3y ago

Yes, it has to in fact. If you have zero context to attend to in a transformer, and you try to predict the first token, you effectively are multiplying a zero-vector by the attention head, making all tokens equally likely in the final softmax (unless the lm_head has a bias, but at least in GPT it does not). So the <|beginning of text|> token, with no context before it, learns to predict the first-token-in-a-document distribution. That's not quite the same as predicting nothing at all.

llwu 3y ago

This is a cool idea - it could be tested it by seeing whether W_V tends to map <bot> to 0, and whether <bot> tends to get attended to. There are interesting things to said whether or not this turns out to be the case.