4 ms·

RAG is much closer to how the human brain works than fine tuning a model is. Arguably, RAG points to the existence or the beginnings of AGI. The computer can ta

RAG is much closer to how the human brain works than fine tuning a model is. Arguably, RAG points to the existence or the beginnings of AGI. The computer can take new information and use it to inform their answer.

phillipcarter 3y ago
> Arguably, RAG points to the existence or the beginnings of AGI. The computer can take new information and use it to inform their answer. I disagree with that. The model isn't fetching information to augment its responses, nor does it have any notion of ground truth or accuracy. I would argue that RAG points at the inverse - we have these incredible probabilistic autocomplete machines but they're so incapable of being reliable that we have to force them to work in a context that we establish for them.
- ldjkfkdsjnv 3y ago
  Right but human beings dont have infinite wells of knowledge. They have to be provided new information to make new decisions. Google searches is RAG for humans. Sure, the model is unable to request the information, but thats the next step.
  - phillipcarter 3y ago
    > Sure, the model is unable to request the information, but thats the next step. How does it decide which information to request? How does it know which is the right or wrong information for a given request? Maybe it's possible to wave this away with "something in the future will do that!" but that's literally the entire reason why all current AI is limited. It doesn't have the business context or reasoning capabilities to make the right decisions to decide what the right data to augment generation even is. I don't believe we're any closer to AGI.
    - jamesbriggs 3y ago
      Not weighing on whether it is AGI-like or not, but my view on the search component is that it's like when we google info, we have a search term and we don't know whether that will return good results or not. So I suppose in order for RAG to become better the LLM needs to be able to review the information given, and decide whether to search more (which they can do with agents to be fair), and we also need to be building retrieval systems that do very well at returning the right information even with sub-optimal search queries. I think LLMs are fairly good at deciding what information is relevant to a query or not though, it is rare that I have found an LLM to be distracted by RAG results (even when they're not relevant)
  - janalsncm 3y ago
    The problem is that facts are not differentiable. If I have a statement “The Declaration of Independence was signed in 1776” and the model replaces 1776 with 1775 that’s incorrect but in some sense just as incorrect as 1876 or 1976. So the loss at all but the correct output should be 1.0, making it pretty hard to learn. It’s like finding a hole in an infinite flat plane with trillions of dimensions.
- esafak 3y ago
  > The model isn't fetching information to augment its responses, nor does it have any notion of ground truth or accuracy. So have it be a separate module? The human brains is compartmentalized too.
janalsncm 3y ago
> RAG is much closer to how the human brain works than fine tuning a model is. Arguably, RAG points to the existence or the beginnings of AGI. I see these two claims as orthogonal. AGI need not work like the human brain, and I’m not sure people’s brains have any lossless representation of memories like a RAG system would. People are infamously unreliable when it comes to recalling information during crimes for example. However, it does seem like a RAG system is much more versatile than continuously training. Facts change all the time and it’s infeasible to retrain every time that happens.
- ldjkfkdsjnv 3y ago
  Isn't RAG analogous to how humans get information during a conversation or searching the internet? Its contextual world information, in the future RAG will also include vision, hearing, sight, the rest of the senses etc. But those are just lower level inputs on what the current language inputs are
  - janalsncm 3y ago
    At a high level yes? There is some representation of facts in the brain, and the same is true in a language model that uses a knowledge base. But if you look any closer the differences are pretty clear. We don’t store facts as a serialized list in our brains. We have a knowledge graph, and a lossy one at that.
dragonwriter 3y ago
> RAG is much closer to how the human brain works than fine tuning a model is. [citation needed] Obviously, the human brain continuously integrates new material, but whether that's more like RAG, continuous fine-tuning, just having a really big context window, or just having a completely different model architecture where activiation (or not) of pathways in inference also produces weight adjustments directly — or doesn't meaningfully work like any of those — is another question, where the answer is less obvious. I think RAG (storing exact data losslessly with some sort of indexing to allow retrieval, and something hidden-prompt-like directing the use of that in populating an inference context) is a lot less likely that something that involves dynamically adjusting model weights. Which also explains forgetting, not just learning.

phillipcarter 3y ago

> Arguably, RAG points to the existence or the beginnings of AGI. The computer can take new information and use it to inform their answer. I disagree with that. The model isn't fetching information to augment its responses, nor does it have any notion of ground truth or accuracy. I would argue that RAG points at the inverse - we have these incredible probabilistic autocomplete machines but they're so incapable of being reliable that we have to force them to work in a context that we establish for them.

ldjkfkdsjnv 3y ago
Right but human beings dont have infinite wells of knowledge. They have to be provided new information to make new decisions. Google searches is RAG for humans. Sure, the model is unable to request the information, but thats the next step.
- phillipcarter 3y ago
  > Sure, the model is unable to request the information, but thats the next step. How does it decide which information to request? How does it know which is the right or wrong information for a given request? Maybe it's possible to wave this away with "something in the future will do that!" but that's literally the entire reason why all current AI is limited. It doesn't have the business context or reasoning capabilities to make the right decisions to decide what the right data to augment generation even is. I don't believe we're any closer to AGI.
  - jamesbriggs 3y ago
    Not weighing on whether it is AGI-like or not, but my view on the search component is that it's like when we google info, we have a search term and we don't know whether that will return good results or not. So I suppose in order for RAG to become better the LLM needs to be able to review the information given, and decide whether to search more (which they can do with agents to be fair), and we also need to be building retrieval systems that do very well at returning the right information even with sub-optimal search queries. I think LLMs are fairly good at deciding what information is relevant to a query or not though, it is rare that I have found an LLM to be distracted by RAG results (even when they're not relevant)
- janalsncm 3y ago
  The problem is that facts are not differentiable. If I have a statement “The Declaration of Independence was signed in 1776” and the model replaces 1776 with 1775 that’s incorrect but in some sense just as incorrect as 1876 or 1976. So the loss at all but the correct output should be 1.0, making it pretty hard to learn. It’s like finding a hole in an infinite flat plane with trillions of dimensions.
esafak 3y ago
> The model isn't fetching information to augment its responses, nor does it have any notion of ground truth or accuracy. So have it be a separate module? The human brains is compartmentalized too.

janalsncm 3y ago

> RAG is much closer to how the human brain works than fine tuning a model is. Arguably, RAG points to the existence or the beginnings of AGI. I see these two claims as orthogonal. AGI need not work like the human brain, and I’m not sure people’s brains have any lossless representation of memories like a RAG system would. People are infamously unreliable when it comes to recalling information during crimes for example. However, it does seem like a RAG system is much more versatile than continuously training. Facts change all the time and it’s infeasible to retrain every time that happens.

ldjkfkdsjnv 3y ago
Isn't RAG analogous to how humans get information during a conversation or searching the internet? Its contextual world information, in the future RAG will also include vision, hearing, sight, the rest of the senses etc. But those are just lower level inputs on what the current language inputs are
- janalsncm 3y ago
  At a high level yes? There is some representation of facts in the brain, and the same is true in a language model that uses a knowledge base. But if you look any closer the differences are pretty clear. We don’t store facts as a serialized list in our brains. We have a knowledge graph, and a lossy one at that.

dragonwriter 3y ago

> RAG is much closer to how the human brain works than fine tuning a model is. [citation needed] Obviously, the human brain continuously integrates new material, but whether that's more like RAG, continuous fine-tuning, just having a really big context window, or just having a completely different model architecture where activiation (or not) of pathways in inference also produces weight adjustments directly — or doesn't meaningfully work like any of those — is another question, where the answer is less obvious. I think RAG (storing exact data losslessly with some sort of indexing to allow retrieval, and something hidden-prompt-like directing the use of that in populating an inference context) is a lot less likely that something that involves dynamically adjusting model weights. Which also explains forgetting, not just learning.