3 ms·

I've been thinking lately of the two tiered reasoner + tools architecture inspired by LangChain, simonw's writing[0] and this is right along those lines. We're

by senko 3y ago

I've been thinking lately of the two tiered reasoner + tools architecture inspired by LangChain, simonw's writing[0] and this is right along those lines. We're trying too hard to have one model do it all. If we coordinate multiple models + other tools (ala ReAct pattern) we could make the systems more resistant to prompt injection (and possibly other) attacks and leverage their respective strengths and weaknesses. I'm a bit wary of tool invocation via python code instead of prompting the "reasoning" LLM to teach it about the special commands it can invoke. Python's a good crutch because LLMs know it reasonably well (I use a similar trick in my project, but I parse the resulting AST instead of running the untrusted code) so it's simpler to prompt them. In a few iterations I expect to see LLMs fine tuned to know about the standard toolset at their disposal (eg. huggingface default tools) and further refinement of the two-tiered pattern. [0] https://simonwillison.net/2023/Apr/25/dual-llm-pattern/ https://simonwillison.net/2023/Apr/25/dual-llm-pattern/

piloto_ciego 3y ago
I’ve been thinking this way too. Our brains have different areas with different functions… so like, why wouldn’t a good AI too? Maybe an LLM for an internal monologue, maybe two or three to debate each other realistically, then a computer vision model to process visual input…
- TeMPOraL 3y ago
  ... and perhaps a LLM that shares the latent space with the visual model, because those are apparently (and surprisingly) easily mapped to each other (at least per that one paper that popped up the other day). The bit that's missing is on-line learning. There's only so much you can keep bouncing around in working memory (context window of all the component models) - eventually you want to "fix" some of the context by altering the weights of the models (a kind of gradual fine-tuning?).
green_man_lives 3y ago
I am not sure if you've ever heard of Julian Jaynes, but you might be interested. His theory of the bicameral mind is pretty interesting. To bastardize it, the unconscious mind is a really good generalized searcher/pattern completer. You can set it loose and it will search some space or optimize some problem and return a solution. The conscious mind is a really good planner, organizer, supervisor, and filter. Basically the conscious mind will come up with a plan and keep track of all the things it has done and needs to do, while the unconscious chugs along solving the problems underneath. The conscious mind will choose to implement the things that the unconscious comes up with based on its discretion (slap that old woman! "Nope"). The conscious mind is good at this because it can sort of simulate the outcome of these "searches" and see what would happen "slapping that old lady would hurt her and get me arrested". So his model sort of sounds like an unrestricted LLM for the unconscious, with another, more restrictive LLM for the conscious, that has access to some sort of crazy deep Q-learning model that can simulate the outcomes of actions taken.

by senko 3y ago

piloto_ciego 3y ago

I’ve been thinking this way too. Our brains have different areas with different functions… so like, why wouldn’t a good AI too? Maybe an LLM for an internal monologue, maybe two or three to debate each other realistically, then a computer vision model to process visual input…

TeMPOraL 3y ago
... and perhaps a LLM that shares the latent space with the visual model, because those are apparently (and surprisingly) easily mapped to each other (at least per that one paper that popped up the other day). The bit that's missing is on-line learning. There's only so much you can keep bouncing around in working memory (context window of all the component models) - eventually you want to "fix" some of the context by altering the weights of the models (a kind of gradual fine-tuning?).

green_man_lives 3y ago

I am not sure if you've ever heard of Julian Jaynes, but you might be interested. His theory of the bicameral mind is pretty interesting. To bastardize it, the unconscious mind is a really good generalized searcher/pattern completer. You can set it loose and it will search some space or optimize some problem and return a solution. The conscious mind is a really good planner, organizer, supervisor, and filter. Basically the conscious mind will come up with a plan and keep track of all the things it has done and needs to do, while the unconscious chugs along solving the problems underneath. The conscious mind will choose to implement the things that the unconscious comes up with based on its discretion (slap that old woman! "Nope"). The conscious mind is good at this because it can sort of simulate the outcome of these "searches" and see what would happen "slapping that old lady would hurt her and get me arrested". So his model sort of sounds like an unrestricted LLM for the unconscious, with another, more restrictive LLM for the conscious, that has access to some sort of crazy deep Q-learning model that can simulate the outcomes of actions taken.