4 ms·
A Large Language Model itself can't perform RAG: a model is a big binary blob of matrices that you run prompts against. Anything that can do RAG is, by definit
by simonw 3y ago
A Large Language Model itself can't perform RAG: a model is a big binary blob of matrices that you run prompts against.
Anything that can do RAG is, by definition, a system that wraps an LLM with additional code that performs the retrieval.
It's the difference between ChatGPT (software that wraps a model and can extra features such as tool usage, Code Interpreter, RAG lookup via Bing etc) and GPT-4 Turbo (a model).
- p1esk 3y agoWhy can’t a model explore its environment (given access) and find and use those tools? Why can’t a model fire up a query (sql, http, or whatever) to do the retrieval if it determines it needs more information?
- arunsr1ni 3y agoThough the terminology is incorrect (a model by itself cannot 'perform' word other than encode/decode, the llm agents are developed to do what you're stating. But when you think about an llm agent doing all sorts of things like sql etc, it effectively becomes a RAG (which in itself is a process than a technology)
- alfor 3y agoDoes the LLM get trained as the database get builded? As you train a model it should build it's knowledge database at the same time. I guess that's how it work? When an LLM get trained on next word guessing, can it search the web to improve his results?
- dlubarov 3y agoI think it's just a matter of semantics - "model" usually refers to a neural network or some similarly pure, deterministic computation, or so I thought.
- simonw 3y agoI'm talking about Large Language Models - the architectures behind most of the current generative text AI boom. In order to use tools they need to be run as part of a system that grants them access to tools, eg via the reAct pattern. https://til.simonwillison.net/llms/python-react-pattern https://til.simonwillison.net/llms/python-react-pattern
- p1esk 3y agoAre you talking about a simple script which executes model’s instructions? It can literally be as simple as connecting the model to my computer command line. I will tell it what I want done and give it my cc number or website creds. Obviously I’m talking about next gen models, like gpt5/6.
- 0x008 3y agoFor that a model needs to recursively parse its output. That is not possible with the current architectures, we need Agent Frameworks for that.