What I want is the LLM to be the last layer of filtering. Basically I want to take the "regular" search results and ask the LLM if they are actually a good matc

What I want is the LLM to be the last layer of filtering. Basically I want to take the "regular" search results and ask the LLM if they are actually a good match, and downrank them if now. So instead of getting a list of pages that contain that error message the LLM can evaluate which ones are closest to my configuration (are they the same OS, are variable parts of the error message the same or similar to mine, does the page have a solution or at least a discussion, or just one person posting the error with no other info). Additionally search engines generally suck at negative phrases. For example I'm looking for some JS library that doesn't depend on React. Current search engines really suck at differentiating between "Foo for React" and "Foo, no React required". LLMs are pretty good at this. That being said LLMs are also imperfect, so I wouldn't want the results that they reject to be completely hidden, but they can be ranked much lower. I think the biggest improvement would come from searches with many candidate pages (like looking for some library that fits my needs) rather than rare ones (like that error that has only been mentioned twice on the internet). But lots of times I am just looking for something with a very specific set of requirements that could be evaluated by an LLM (with pretty good accuracy). That can greatly improve my experience compared to getting all textually matching searches and doing the filtering myself.