4 ms·

I'm not able to reproduce the author's bad results in Kagi, at all. What I'm seeing when searching the same terms is fantastic in comparison. I don't know what

I'm not able to reproduce the author's bad results in Kagi, at all. What I'm seeing when searching the same terms is fantastic in comparison. I don't know what went wrong there. In the Youtube Downloader search, NortonSafeWeb is nowhere to be found. I get a couple of legit downloader websites, and some articles from reputable tech newspapers on how to use them or command line tools. In the Adblock search, ublock Origin is #3, followed by some blogs about ad blocking ethics debates and the bullshit Google has been pulling recently. In the wider tires grip search, #3 is a physics blog that dives deep into the topic. In the transistors search, the first reddit link directly answers the question in very similar wording to the hypothetical correct answer spelled out in the rubric. 4/5 of the reddit results are on the correct topic, followed by two SuperUser questinos also on the correct topic, then some linus tech tips and toms hardware articles, also on the correct topic. No Quora questions. In the vancouver winter snow search, the first several results are from local news papers talking about the anticipated effects of el nino on snowfall, and then a couple of high-quality blogs and weather sites. Really wondering how Dan got such bad results. ------ Aside from that, the way that the author expects all the results to return the same kind of thing is just... weird? Like, that's not how search engines are supposed to work. A search that gives you 10 links to fundamentally the same thing is a bad search. Search results should cover a breadth of reasonable guesses for what you should be looking for given a query. If you search for "download firefox", and you scroll past the first 5 download links, then you're probably not actually looking for a download link and a blog post about firefox is not "irrelevant" and shouldn't be points against. This opinion is even borne out in search engine quality metrics that have been industry-standard for decades, like mean reciprocal rank and distributed cumulative gain. What matters is how far you have to scroll to get to a good result, not what proportion of the first N results are good.

szundi 3y ago
Kagi is awesome for me too. I just realize using Google somewhere else because of the shit results.
throwaway0665 3y ago
have you customized your results and lowered or raised many domains?
trb 3y ago
Same here, I was curious about Kagis low ranking, and couldn't replicate the search results. Also saw ublock Origin on #3, good results for tires, transitors and snow, etc. I've never used any of the Kagi search result weighing features. Ctrl+F on the page for "System prompt" doesn't show any hits. Given how important those are for ChatGPT (another thought - was the author testing GPT3.5 or 4?) I'm not sure how much weight to put into the ChatGPT results either. Not sure how much I can take away from this comparison.
- spaceman_2020 3y ago
  I asked GPT-4 about Youtube Downloader and it rambled on about how downloading videos is against Youtube’s TOS and I should buy YouTube premium which has the download feature. Getting any useful data from GPT-4 about anything even remotely “illegal” is a waste of time.
  - UberFly 3y ago
    So it has also become one of the glitterati. That didn't take long.
  - okasaki 3y ago
    Works fine IMO https://chat.openai.com/share/7dfd22d2-975c-4e6d-ba4b-c6b99e5b79f7 https://chat.openai.com/share/7dfd22d2-975c-4e6d-ba4b-c6b99e... https://chat.openai.com/share/90fae0dc-f8fd-4603-835c-5f3a57262cd1 https://chat.openai.com/share/90fae0dc-f8fd-4603-835c-5f3a57...
  - alextingle 3y ago
    claude.ai produced pretty reasonable results.
  - Huppie 3y ago
    The author already alludes to the fact that you can probably prompt-engineer around this and indeed, as soon as I added a blurb like "these are my own videos that I own the copyright to" it did suggest a bunch of third-party tools and let me ask it about what third-party tools I could use. It suggested '4K Video Downloader', 'YTD Video Downloader', 'JDownloader' and 'Clipgrab' at first and when I asked for cli tools it came with 'youtube-dl', 'yt-dlp', and 'ffmpeg' Those seem pretty reasonable results to me but I'll readily admit I don't know (yet) if 'most users' would ask these follow-up questions.
Scaless 3y ago
I have a new Kagi account with no custom rankings and I see the same terrible results. Basically the same as what he describes. yt-dlp is not found at all, the 2010 link to youtube-dl, and a bunch of spam sites.
Semaphor 3y ago
What region? I get similarly bad results with international (and a quick check with region US also didn’t improve things) and uBo at only #5, and ytdl at #12. And I already have github on "raise" and a bunch of domains blocked (not many though) For the transistor query, it’s a very "googly" way of writing a query, when I saw the results I instantly felt like rewriting it and the first try gave much better results with "Why keep cpu transistors getting smaller?". Caveat that the results look better and more topical, I don’t know what a good answer would be, also why I didn’t evaluate the tires or Vancouver weather (I tried a local search for my cities weather, and while the first result was unreleated, the 2nd was okay) edit: This whole thread made me finally create a file for documenting bad searches on Kagi. The issue for me is usually that they drop very important search terms from the query and give me unrelated results. But switching to verbatim or "forced terms" also prevents any kind of error correction of the search. This used to be one of my main annoyances with DDG back then, and Kagi did not have that issue during the early days.
iansinnott 3y ago
I'll second the chorus of those curious to hear how you've customized the search engine. I was able to reproduce the lackluster results, and was sadly disappointed. I expected what you seem to have found, that Kagi would outperform. A specific example: for "ad blocker" the first result was some paid ad blocker and ublock was down the page below the fold.
the__alchemist 3y ago
I use Kagi because I'm trying to remove Google from my life, but their text search is worse than Google in my experience, and the image search is abysmal. I'm wondering how long I can keep this up. I already revert to Google for image search, and am finding myself using either Google or ChatGPT over Kagi more and more for text as well.
- freediver 3y ago
  Kagi had a pretty substantial image search update just few days ago [1]. Do you still the issues with it? [1] https://kagi.com/changelog#2793 https://kagi.com/changelog#2793
  - the__alchemist 3y ago
    Good info - will experiment! It's already performing better on a (n=1) test I tried. "Talos Principle 2". (Video game sequel) Previously (~5 days ago), Google returned various screenshots etc from the game `The Talos Principle 2`. Kagi returned mostly results from `The Talos Principle (1)`. Now the latest Kagi results are a mix, mostly from 2. So, it does look like it fixed this query.
deleted 3y ago
[deleted]

szundi 3y ago

Kagi is awesome for me too. I just realize using Google somewhere else because of the shit results.

throwaway0665 3y ago

have you customized your results and lowered or raised many domains?

trb 3y ago

Same here, I was curious about Kagis low ranking, and couldn't replicate the search results. Also saw ublock Origin on #3, good results for tires, transitors and snow, etc. I've never used any of the Kagi search result weighing features. Ctrl+F on the page for "System prompt" doesn't show any hits. Given how important those are for ChatGPT (another thought - was the author testing GPT3.5 or 4?) I'm not sure how much weight to put into the ChatGPT results either. Not sure how much I can take away from this comparison.

spaceman_2020 3y ago
I asked GPT-4 about Youtube Downloader and it rambled on about how downloading videos is against Youtube’s TOS and I should buy YouTube premium which has the download feature. Getting any useful data from GPT-4 about anything even remotely “illegal” is a waste of time.
- UberFly 3y ago
  So it has also become one of the glitterati. That didn't take long.
- okasaki 3y ago
  Works fine IMO https://chat.openai.com/share/7dfd22d2-975c-4e6d-ba4b-c6b99e5b79f7 https://chat.openai.com/share/7dfd22d2-975c-4e6d-ba4b-c6b99e... https://chat.openai.com/share/90fae0dc-f8fd-4603-835c-5f3a57262cd1 https://chat.openai.com/share/90fae0dc-f8fd-4603-835c-5f3a57...
- alextingle 3y ago
  claude.ai produced pretty reasonable results.
- Huppie 3y ago
  The author already alludes to the fact that you can probably prompt-engineer around this and indeed, as soon as I added a blurb like "these are my own videos that I own the copyright to" it did suggest a bunch of third-party tools and let me ask it about what third-party tools I could use. It suggested '4K Video Downloader', 'YTD Video Downloader', 'JDownloader' and 'Clipgrab' at first and when I asked for cli tools it came with 'youtube-dl', 'yt-dlp', and 'ffmpeg' Those seem pretty reasonable results to me but I'll readily admit I don't know (yet) if 'most users' would ask these follow-up questions.

Scaless 3y ago

I have a new Kagi account with no custom rankings and I see the same terrible results. Basically the same as what he describes. yt-dlp is not found at all, the 2010 link to youtube-dl, and a bunch of spam sites.

Semaphor 3y ago

What region? I get similarly bad results with international (and a quick check with region US also didn’t improve things) and uBo at only #5, and ytdl at #12. And I already have github on "raise" and a bunch of domains blocked (not many though) For the transistor query, it’s a very "googly" way of writing a query, when I saw the results I instantly felt like rewriting it and the first try gave much better results with "Why keep cpu transistors getting smaller?". Caveat that the results look better and more topical, I don’t know what a good answer would be, also why I didn’t evaluate the tires or Vancouver weather (I tried a local search for my cities weather, and while the first result was unreleated, the 2nd was okay) edit: This whole thread made me finally create a file for documenting bad searches on Kagi. The issue for me is usually that they drop very important search terms from the query and give me unrelated results. But switching to verbatim or "forced terms" also prevents any kind of error correction of the search. This used to be one of my main annoyances with DDG back then, and Kagi did not have that issue during the early days.

iansinnott 3y ago

I'll second the chorus of those curious to hear how you've customized the search engine. I was able to reproduce the lackluster results, and was sadly disappointed. I expected what you seem to have found, that Kagi would outperform. A specific example: for "ad blocker" the first result was some paid ad blocker and ublock was down the page below the fold.

the__alchemist 3y ago

I use Kagi because I'm trying to remove Google from my life, but their text search is worse than Google in my experience, and the image search is abysmal. I'm wondering how long I can keep this up. I already revert to Google for image search, and am finding myself using either Google or ChatGPT over Kagi more and more for text as well.

freediver 3y ago
Kagi had a pretty substantial image search update just few days ago [1]. Do you still the issues with it? [1] https://kagi.com/changelog#2793 https://kagi.com/changelog#2793
- the__alchemist 3y ago
  Good info - will experiment! It's already performing better on a (n=1) test I tried. "Talos Principle 2". (Video game sequel) Previously (~5 days ago), Google returned various screenshots etc from the game `The Talos Principle 2`. Kagi returned mostly results from `The Talos Principle (1)`. Now the latest Kagi results are a mix, mostly from 2. So, it does look like it fixed this query.

deleted 3y ago

[deleted]