5 ms·
No method today is a panacea. RAG naturally isn’t the approach you would take if you’re trying to summarize a large body of text, as in your example. That doe
by aik 3y ago
No method today is a panacea. RAG naturally isn’t the approach you would take if you’re trying to summarize a large body of text, as in your example. That doesn’t make sense. That is not what it is for.
What it’s awesome for is asking a somewhat specific question about some thing in a book if it can be answered by reviewing a paragraph or a few across the entire book.
So to answer your question: Start by using the right tool for the job. The problem with this is: Some tools are prohibitively expensive, challenging, or very time-consuming to use.
- deegles 3y agoBut the issue remains where your questions have to have easy "retrieval" aspects. You can't ask general questions that a human would have no trouble with. A book report is the example I used because you need to have read the entire book. essentially a book-length context window.
- namibj 3y agoThankfully, there are methods (like causal transformers, but also some question/text/answer architectures) for not re-computing self-attention between the document/knowledge-base tokens across the iterations of autoregressive decoding for generating the response.
- aik 3y agoDo you have an example of where these methods still produce good summaries? Eg if you adjust how re-computation of self-attention in autoregressive decoding / between token generations works to significantly decrease the amount of computation needed?
- bradknowles 3y agoThe real problem is knowing what tools exist for the job. Or that the right tool actually does exist. I would submit this is true for many fields of knowledge.