3 ms·

But here is the thing, if you rely on RAG or any other knowledge injection for recommendation, you essentially make your LLM parrot machine, no better than a se

But here is the thing, if you rely on RAG or any other knowledge injection for recommendation, you essentially make your LLM parrot machine, no better than a search engine, just much slower. Knowledge is one aspect of it, I found its instruction following ability, frustrating as well. I think Ilya Sutskever puts it very well, larger model brings stability to wider range of tasks, what smaller models are not capable of. Even though book recommendation with GPT might a niche, but it is not something really unexpected TBH, thus comes my disappointment.

chasd00 3y ago
> But here is the thing, if you rely on RAG or any other knowledge injection for recommendation, you essentially make your LLM parrot machine, no better than a search engine, just much slower. this is a really good point. I was working on some careful q/a data curation today that is then fed to a vector store where embeddings are calculated and served. I realized that my carefully curated q/a data in combination with the vector database works just fine for what i want to do all by itself. A really good semantic search of my q/a database turns up answers to my questions with no rag llm prompting required. When I added the llm it just put the same information in different words, not super useful when i could have just looked at the returned embeddings and gotten the same information.
- a_wild_dandan 3y ago
  It sounds like you used the wrong tool for the job, then blamed the tool.
avereveard 3y ago
You can still leverage the llm ability to understand language to do better than search. You can use it to refine or expand your search terms, you can ask the llm to ask you questions about what book you'd like next and convert your answers into genre/theme/setting/mood to feed into the next step of the pipeline, you can tell the agent to narrow search results by asking you partitioning questions from the list of book the search returned instead of just being a dry ranking The is so much rag can do if you use the llm part properly. If the rag pipeline goes question > embedding > search > summarization then yeah you're getting an expensive slow parrot no better than just searching, but that is because it's using the baseline rag that uninspired consultant describe in blogs in a scramble to position themselves as "expert" in the new market.

chasd00 3y ago

> But here is the thing, if you rely on RAG or any other knowledge injection for recommendation, you essentially make your LLM parrot machine, no better than a search engine, just much slower. this is a really good point. I was working on some careful q/a data curation today that is then fed to a vector store where embeddings are calculated and served. I realized that my carefully curated q/a data in combination with the vector database works just fine for what i want to do all by itself. A really good semantic search of my q/a database turns up answers to my questions with no rag llm prompting required. When I added the llm it just put the same information in different words, not super useful when i could have just looked at the returned embeddings and gotten the same information.

a_wild_dandan 3y ago
It sounds like you used the wrong tool for the job, then blamed the tool.

avereveard 3y ago

You can still leverage the llm ability to understand language to do better than search. You can use it to refine or expand your search terms, you can ask the llm to ask you questions about what book you'd like next and convert your answers into genre/theme/setting/mood to feed into the next step of the pipeline, you can tell the agent to narrow search results by asking you partitioning questions from the list of book the search returned instead of just being a dry ranking The is so much rag can do if you use the llm part properly. If the rag pipeline goes question > embedding > search > summarization then yeah you're getting an expensive slow parrot no better than just searching, but that is because it's using the baseline rag that uninspired consultant describe in blogs in a scramble to position themselves as "expert" in the new market.