4 ms·
You forgot all the inner monologue (https://www.gwern.net/docs/ai/gpt/inner-monologue/index https://www.gwern.net/docs/ai/gpt/inner-monologue/index) & scratchpa
by gwern 5y ago
You forgot all the inner monologue (https://www.gwern.net/docs/ai/gpt/inner-monologue/index https://www.gwern.net/docs/ai/gpt/inner-monologue/index) & scratchpad papers which give it additional steps or access to Python REPL etc: eg https://arxiv.org/abs/2112.15594 https://arxiv.org/abs/2112.15594 https://arxiv.org/abs/2111.08267 https://arxiv.org/abs/2111.08267 https://arxiv.org/abs/2111.08171 https://arxiv.org/abs/2111.08171
- visarga 5y agoAI Chains really takes it to the next level.
- gwern 5y agoYeah, but I didn't bring it up because I wasn't sure how much is really the model choosing and how much is the human workflow: they emphasize the interactive part heavily. Anyway, today another great paper dropped on self-distillation: "STaR: Bootstrapping Reasoning With Reasoning" https://arxiv.org/abs/2203.14465 https://arxiv.org/abs/2203.14465 , Zelikman et al 2022.
- deleted 5y ago[deleted]