3 ms·

Maybe I’m interpreting this differently but to me modern LLM+full text search means “agentic” - LLM gets to pick the search terms and iterate on them. The under

by andy99 1mo ago

Maybe I’m interpreting this differently but to me modern LLM+full text search means “agentic” - LLM gets to pick the search terms and iterate on them. The underlying LLM does know synonyms etc, better and more flexibly than an embedding model, and gets explainable feedback from failed searches.

mdp2021 1mo ago
That could work in a way, but it's very expensive as expressed and I do not know of prominent robust implementations. On the other hand, your post may contain a good idea: L=instruct_LLM("provide a list of synonyms and periphrases of terms T within context C", T, C); then iter(`grep l in L`). One NN query and a `grep` collection. But again, if one wanted to order the results, it is either through a dumb crierion or through another LLM query - but this could make it extremely costly (requiring either a huge context or a quadratic number of ordering queries). And, the above `grep` based procedure would remain keyword based and not semantic based, which means that the user must know that it will not be based on comprehension but on the possible results that keyword matching can yield.
- woah 1mo ago
  You do not know of prominent robust implementations? This is how Claude Code, GPT Codex, etc have worked for a couple years. And they do tend to be impressively good at navigating large amounts of text.
  - mdp2021 1mo ago
    Thank you, no, I did not know that. Where have you found the info? Sebastian Raschka, Anthropic/OpenAI blogs? (BTW: you made me realize - I had to take "time off" for over half a year... I am sure I missed a lot.) -- Edit: for clarity: for "full text search" we remain on the interpretation of "searching for literal substrings" - and whether plain user provided keywords list or LLM enriched list based on the former, and whether more or less successful, it remains a syntactic search quite distinct from a semantic one. Having an LLM enrich the original keywords list can be a good idea, but the possibility of misses remains when compared to a properly working semantic search.
    - woah 1mo ago
      Yes, they navigate with heavy use of the "grep" tool
    - antonvs 1mo ago
      > Where have you found the info? You can see it in action if you watch the “chain of thought” text when using coding agents.

by andy99 1mo ago

mdp2021 1mo ago

That could work in a way, but it's very expensive as expressed and I do not know of prominent robust implementations. On the other hand, your post may contain a good idea: L=instruct_LLM("provide a list of synonyms and periphrases of terms T within context C", T, C); then iter(`grep l in L`). One NN query and a `grep` collection. But again, if one wanted to order the results, it is either through a dumb crierion or through another LLM query - but this could make it extremely costly (requiring either a huge context or a quadratic number of ordering queries). And, the above `grep` based procedure would remain keyword based and not semantic based, which means that the user must know that it will not be based on comprehension but on the possible results that keyword matching can yield.

woah 1mo ago
You do not know of prominent robust implementations? This is how Claude Code, GPT Codex, etc have worked for a couple years. And they do tend to be impressively good at navigating large amounts of text.
- mdp2021 1mo ago
  Thank you, no, I did not know that. Where have you found the info? Sebastian Raschka, Anthropic/OpenAI blogs? (BTW: you made me realize - I had to take "time off" for over half a year... I am sure I missed a lot.) -- Edit: for clarity: for "full text search" we remain on the interpretation of "searching for literal substrings" - and whether plain user provided keywords list or LLM enriched list based on the former, and whether more or less successful, it remains a syntactic search quite distinct from a semantic one. Having an LLM enrich the original keywords list can be a good idea, but the possibility of misses remains when compared to a properly working semantic search.
  - woah 1mo ago
    Yes, they navigate with heavy use of the "grep" tool
  - antonvs 1mo ago
    > Where have you found the info? You can see it in action if you watch the “chain of thought” text when using coding agents.