6 ms·
There are many papers trying to couple language models with external modules. In the Retrieval-Enhanced Transformer (RETRO) paper a large language model was co
by visarga 5y ago
There are many papers trying to couple language models with external modules.
In the Retrieval-Enhanced Transformer (RETRO) paper a large language model was coupled with a similarity based text index. It can populate the prompt with relevant information from the index thus being more grounded and update-able.
In another paper (AlphaCode) the language model was coupled with a compiler and could run programs and check if they match the expected outputs for a few test cases. The model was able to solve competition style coding problems above average human score.
In another paper (Language Models as Zero Shot Planners) a language model generates commands to navigate a virtual home environment and performs tasks. The knowledge in the LM helps in quickly learning tasks.
A recent one can learn new concepts by simple conversation, then apply them where necessary. You can talk-train your model. (Memory assisted prompt editing to improve GPT 3 after deployment)
So the trend is to add "toys" on language models - a simulator, a compiler, a search engine, a long term memory module.
I'd like to see a recursive language model, that can sub-call itself to decompose problems.
- gwern 5y agoYou forgot all the inner monologue (https://www.gwern.net/docs/ai/gpt/inner-monologue/index https://www.gwern.net/docs/ai/gpt/inner-monologue/index) & scratchpad papers which give it additional steps or access to Python REPL etc: eg https://arxiv.org/abs/2112.15594 https://arxiv.org/abs/2112.15594 https://arxiv.org/abs/2111.08267 https://arxiv.org/abs/2111.08267 https://arxiv.org/abs/2111.08171 https://arxiv.org/abs/2111.08171
- visarga 5y agoAI Chains really takes it to the next level.
- gwern 5y agoYeah, but I didn't bring it up because I wasn't sure how much is really the model choosing and how much is the human workflow: they emphasize the interactive part heavily. Anyway, today another great paper dropped on self-distillation: "STaR: Bootstrapping Reasoning With Reasoning" https://arxiv.org/abs/2203.14465 https://arxiv.org/abs/2203.14465 , Zelikman et al 2022.
- deleted 5y ago[deleted]
- sprobertson 5y ago> I'd like to see a recursive language model, that can sub-call itself to decompose problems. I tried a very simple and specific version of this a few years ago (Recursive Application of Recurrent Neural Networks) and it worked great for intent parsing: https://github.com/spro/RARNN https://github.com/spro/RARNN Would like to see what "real" researchers with more modern models could do with the concept.
- The_rationalist 5y ago> A recent one can learn new concepts by simple conversation, then apply them where necessary. You can talk-train your model. Which model? Sauce please
- visarga 5y ago"Memory assisted prompt editing to improve GPT 3 after deployment" paper: https://arxiv.org/abs/2201.06009 https://arxiv.org/abs/2201.06009 video: https://www.youtube.com/watch?v=gYxJEd3EUKs https://www.youtube.com/watch?v=gYxJEd3EUKs
- josefx 5y ago> The model was able to solve competition style coding problems above average human score. I am not sure if I am thinking of the right study, but as far as I remember the model included a human wading through and filtering solutions and while there may have been a compiler attached they also scored themselves. The marketing blurb of course tried to make it sound as if they had competed.
- visarga 5y agoThe model generates a large number of solutions, then they filter those that actually compile and generate the right output when executed, then they cluster to select a few (<10 solutions) and submit them. They are not allowed to present too many attempts. Here's a good analysis of the paper: https://www.youtube.com/watch?v=s9UAOmyah1A https://www.youtube.com/watch?v=s9UAOmyah1A
- josefx 5y agoAh, the paper describes a fixed method for the last selection step and also AI generated tests to reduce the results even more before that. Quite a bit better, even if the participation is still only simulated.