3 ms·
I think it's fairly simple: you're creating space for intermediary tokens to be generated, where those intermediary tokens represent "thoughts" or a simulated i
by wnmurphy 3y ago
I think it's fairly simple: you're creating space for intermediary tokens to be generated, where those intermediary tokens represent "thoughts" or a simulated internal dialog.
Without that, it's analogous to asking someone a question and they immediately start responding from some information they'd heard before, rather than taking some time to have an inner dialog with themself.
- kelseyfrog 3y agoThere's a recent paper which seeks to explicitly perform time-to-think using pause tokens[1]. > However sophisticated this end-to-end process may be, it abides by a peculiar constraint: the number of operations determining the next token is limited by the number of tokens seen so far. There are obviously pros and cons to each, but nothing excludes us from combining the two either. 1. Think before you speak: Training Language Models With Pause Tokens https://arxiv.org/abs/2310.02226v2 https://arxiv.org/abs/2310.02226v2