4 ms·
The problem is not the short term context length, but long term. If you want to do things like long term goal planning, keeping track of distant past events can
by jerpint 4y ago
The problem is not the short term context length, but long term. If you want to do things like long term goal planning, keeping track of distant past events can be of high value
- PaulHoule 4y agoI’d argue for planning that the answer is to couple the LLM to some other system rather than try to improve the LLM. Combinatorial optimization is a well-understood problem that is NP-complete in theory but practical to solve in practice. For doing math we might as well give the LLM a pocket calculator, why not couple it to a planner, theorem prover and similar tools?
- pixl97 4y agoThis is pretty much what people are testing now with Auto-GPT
- pmoriarty 4y ago"For doing math we might as well give the LLM a pocket calculator, why not couple it to a planner, theorem prover and similar tools?" ChatGPT has already been hooked up to Wolfram Alpha.[1] For hooking up other things, see HuggingGPT and TaskMatrix.AI.[2][3] [1] - https://writings.stephenwolfram.com/2023/03/chatgpt-gets-its-wolfram-superpowers/ https://writings.stephenwolfram.com/2023/03/chatgpt-gets-its... [2] - https://arxiv.org/pdf/2303.17580.pdf https://arxiv.org/pdf/2303.17580.pdf [3] - https://arxiv.org/pdf/2303.16434.pdf https://arxiv.org/pdf/2303.16434.pdf
- btbuildem 4y agoPrecisely. LLMs are the Perl of the future
- sharemywin 4y agoThat's the interesting thing. if LLM start to just become interfaces for other systems you could probably just use them to train smaller systems like in the style of alpaca and other recent LLMs.
- taneq 4y agoMaybe that’s what our inner monologue is for… to cycle relevant context through regularly so it stays in scope. I mean, that’s not actually why we have one but it’d be cute.