8 ms·

I'm beginning to suspect LLMs' viability for search purposes is dependent on your existing search habits. 40% of my search queries are just copy-pasted error me

• carlosjobim 2y ago
  You should use the right tool for the right purpose. If you're searching for a physical place and want opening hours, reviews and photos, it's Google Maps. If you're looking for updates and photos for a physical place, it's Instagram. If it's detailed articles it's Wikipedia. If it's to define or translate a word, right click and "Look up". Or if it's general search, you have Kagi.
  • ryukoposting 2y ago
    My purpose is Search, and I want one right tool for it. I have had one right tool for my entire life - first Google, and then StartPage since ~2017. I considered Kagi. I like the concept, and I'm willing to pay for search, but Kagi costs more than I'm willing to pay.
    • dartos 2y ago
      FWIW, kagi uses bing under the hood.
      • tombert 2y ago
        I don't think Kagi uses bing anymore: https://news.ycombinator.com/item?id=36530936 https://news.ycombinator.com/item?id=36530936
  • UDontKnowJack 2y ago
    Wikipedia? Are you kidding?
    • ricardo81 2y ago
      wiki is adequate most of the time for proper nouns.
    • lisper 2y ago
      What would you suggest instead?
    • carlosjobim 2y ago
      It's a good starting point.
• TremendousJudge 2y ago
  I'm mostly on the same boat as you, but yes, I'd say most people type complete sentences (usually questions) into Google. The results for these types of queries are usually: - An autogenerated info card that usually is regurgitating wikipedia. - Several SEO blog spam posts that have the exact query as the title and kinda answer it while trying to sell you something. No sources for more info of course, just a link to the store or whatever. With this landscape, LLMs for search kinda make sense. In a way we are already 90% of the way there.
• Al-Khwarizmi 2y ago
  I think the point is not what people type, but what people need... I haven't been typing complete sentences into Google because I know it's not the best way to use Google, but indeed many of the information needs I use google for are very often questions, even if I never typed the question directly but instead tried to find some suitable keywords. LLMs are indeed not the right tool for finding opening hours or phone numbers (at least for now), but I actually find them more useful than Google for error messages. I guess this depends a lot on what compiler/program is reporting the errors.
• nprateem 2y ago
  "I want to write EDM like David Guetta. Suggest where to start, books, etc for someone who understands music theory" Good luck wading through whatever Google gives back for that. Also I find they do well on error messages in general.
  • yoz-y 2y ago
    The results for this particular query are quite okay actually, a relatively recent reddit thread, a quora page, then some forum and youtube results. Funnily, this very HN post is on the first page of results as well.
• martinald 2y ago
  LLMs open a whole new 'search' up for me, when you are not sure what you are searching for. For example, if you can't remember the name of a lib but you know roughly what it does (or if you have misremembered the name of it). This is often very time consuming. It's also far better than Google for recommending things (really anything) as you can use much more "precise" requirements for the recommendations and correct them, instead of getting at best some SEO spam. It's obviously not good for stuff that requires real time updates like opening times, though I suspect with time OpenAI et al will combine their own search index + RAG to solve a lot of this. The elephant in the room though is this is long term going to kill the incentive for a lot of content to be published. AI overviews have definitely reduced the amount of clicking through I do, even though I try and subconsciously 'support' sites by trying to find a link. But often it's right there.
• loudmax 2y ago
  There are some tasks that I would have used a search engine for in the past, because that was more or less the only option. Say, searching for "ffmpeg transcode syntax" and then spending 15 minutes comparing examples from Stack Overflow with the official documentation to try to make sense of them. Now I can tell Claude exactly what I'm trying to accomplish and it will give me an answer in 30 seconds that's either correct, or close enough for me to quickly make up the difference. I'm still going to turn to Google to find out what a store's opening hours or phone number is, as well as a lot of other tasks. But there are types of queries that are better suited for an LLM, that previously could only be done in a search engine. There's also a non-technical reason for LLM search. Google built its business on free search, paid for by advertising, which seemed like a good idea in the early 2000's. A few decades later and we have a better appreciation for the value of the ad-driven business model. Right now, there's a whole lot of money being thrown at online LLMs, so for the most part they're not really doing ads yet. It's refreshing to make a query and not have sponsored results at the top of the list. Obviously, the free online LLM business model isn't going to last indefinitely. In the pretty near future, we'll either need to start paying a usage fee, or parse through advertisements delivered by LLMs as well. But it's nice while it lasts.
  • kevindamm 2y ago
    Ads delivered via LLMs will cost more to distribute, which means higher cost for the businesses purchasing ads, perhaps high enough to deter a lot of smaller ads customers, so I think we'll see an interesting dynamic appear there. Especially if the ad-laden SEO-boosted sites suffer from further enshittification of Search, which has been spinning in its own vicious cycle lately.
  • IG_Semmelweiss 2y ago
    >>>Obviously, the free online LLM business model isn't going to last indefinitely. I think this is one of those points where LLMs have already changed the paradigm People liked being able to search, but would not pay for it. For many queries , the value wasnt there: users still had to scroll thru pages or tinker for the right query for the required result. Eventually search turned so much worse by seo spam, that kagi stepped in to fill the void. LLMs start from a different direction. The value is clearly there. OpenAI etc still have a ton of paying subscribers. I do think eventually some will incorporate ads, but I think innovation has revealed that theres a market -perhaps a substantial one- for fee-based information search with LLMS
• bfeynman 2y ago
  Similar experience. Whenever I have questions where I know what answer should be (i.e. errors, how to do something etc, ELI5) I will use LLMs. When I use google now I almost never use the AI overview because I am almost exclusively trying to discover something that I want from a source. There's also a whole interesting category of search called navigational queries where there is low entropy (and thus ad value) when someone just goes to google to search espn.com, because they will almost always not explore. I do not understand why people would use perplexity or anything over just chatgpt,claude etc unless some price thing - because it occupies this middle latent space of search I don't find useful.
• bambax 2y ago
  I have never typed a complete sentence into Google and am still very satisfied with the quality of Google search. But... your comment made me realize, a good use of LLMs with search would be not to ask them directly, but to use them as a front: detect what the user wants and devise the perfect Google search to find it. LLMs should be very good at discriminating between a plain error message, a business name, or a general open question, building an excellent Google search for it, and parse and interpret the results. It seems that's what Perplexity is doing, mostly. Personally I find it slower and more cumbersome than using Google Search directly, but maybe that's the direction we're all going. Machines to help us use machines.
• hansvm 2y ago
  It's another tool. Learn to use it, and learn when it shouldn't be used. Google abandoned the ability to use most search qualifiers effectively, so for niche websites I'll get zero results even with perfect exact-string-match queries, even when the site is in Google's index. On the other hand if I only vaguely remember the content and no related words or synonyms, Google is unable to turn my fuzzy feelings into an accurate search result. Plus, your ability to filter the "kind" of site is basically non-existent, making it all but impossible to find information on topics vaguely related to any word vaguely related a product you could potentially buy. LLMs hallucinate and have their own set of problems, but that orthogonality makes them very useful situationally. Not too long ago I needed to track down the blog Google references internally in the design doc of their TGIF employee voting platform, regarding Wilson scoring (using confidence intervals instead of means/medians/...). That's very easy to do in Google search if you can remember the right keywords from a decade ago (like Wilson scoring), but otherwise it's impossible. Reframing that problem for an LLM, you'd use plain English to describe everything you know, add a bit of flattery to shift the output distribution to that corner of the internet which actually knows what you're asking, potentially add one sentence to stop this last batch of models from wasting their time actually searching the web, and ask for a list of the top 5 authors and blog titles they think might be correct. That gives you a whole new set of search terms to finish your quest (in my case, the right answer was always in that list of 5, no matter how many times I reran the query). That property of having to add additional context (e.g., when asking for recipes, I'll describe the background of who the LLM is roleplaying first) to get a good result is annoying. Full sentences with proper punctuation, unfortunately, also help. I wrote a small tool to make it easier for me to keep track of prompts I found useful and execute them with modifications. As some other commenters mention, the LLM can help you with XY problems. You're searching for recipes, techniques, nutrition spreadsheets, ..., trying to craft something that meets some set of constraints (e.g., for some hypothetical set of guests you might require: no pork, most dishes have to be vegetarian, most dishes have to be gluten-free, it's fine if cooks all day so long as that isn't active prep time, you'd prefer to make it as tasty as possible while leaning in to cheaper, homestyle cooking, it has to use these red bell peppers I have already, and nutrition doesn't really matter). That's a nontrivial collection of tasks with a proper search engine, almost all of which the LLM is piss-poor at individually, unless you already have a good idea of the kind of dish you want to make. However, if you in two phases ask the LLM to brainstorm a list of 20 meal ideas and then expand on your favorite (that would be a decent time to add any modifications) then that collection of tasks gets done all at once. You _have_ to be able to look at the recipe and decide if it's any good or not, so a beginner probably shouldn't do that, but for everyone else it's a huge time saver. I mentioned that Google sucks at filtering the "kind" of website you'd like to visit. LLMs handle that great. Like always, you have to be able to handle hallucinations (in this case, by just going to the results and checking if they're any good), but consider a prompt like the following: > The web nowadays has tons of ad-infested, profit-driven, barely legible SEO drivel -- even in the top 100 search results and even from huge sites -- but all the old websites like Sheldon Brown's wealth of bicycle knowledge are still out there. List the top three old-web resources I'd want to read to learn about grafting apple trees. When I ran that, I got one commercial result, one journal with a wealth of paywalled information, and one forum with a huge collection of free information about the particular species of Apple I'm interested in. For "apple grafting" in particular, Google does just fine (a bit better arguably since the 1st Google result was the 2nd LLM result, and it's the only one I really cared about), but the more commercialized the knowledge you're looking for is the more the LLM shines out in comparison.
• rustcleaner 2y ago
  >Have you people been typing complete sentences into Google all these years? Well somebody has to be responsible for promoting the cancer which converted search boxes from logical set membership definition strings to "Smart™ boxes," designed to piss you off and then direct you to a sponsored product page.
• kevincox 2y ago
  What I want is the LLM to be the last layer of filtering. Basically I want to take the "regular" search results and ask the LLM if they are actually a good match, and downrank them if now. So instead of getting a list of pages that contain that error message the LLM can evaluate which ones are closest to my configuration (are they the same OS, are variable parts of the error message the same or similar to mine, does the page have a solution or at least a discussion, or just one person posting the error with no other info). Additionally search engines generally suck at negative phrases. For example I'm looking for some JS library that doesn't depend on React. Current search engines really suck at differentiating between "Foo for React" and "Foo, no React required". LLMs are pretty good at this. That being said LLMs are also imperfect, so I wouldn't want the results that they reject to be completely hidden, but they can be ranked much lower. I think the biggest improvement would come from searches with many candidate pages (like looking for some library that fits my needs) rather than rare ones (like that error that has only been mentioned twice on the internet). But lots of times I am just looking for something with a very specific set of requirements that could be evaluated by an LLM (with pretty good accuracy). That can greatly improve my experience compared to getting all textually matching searches and doing the filtering myself.