3 ms·

> from Reddit threads Google is the only search engine allowed to index Reddit [1]. [1] https://www.lifewire.com/google-reddit-deal-8685766 https://www.lifewi

by nomel 8mo ago

> from Reddit threads Google is the only search engine allowed to index Reddit [1]. [1] https://www.lifewire.com/google-reddit-deal-8685766 https://www.lifewire.com/google-reddit-deal-8685766

Aurornis 8mo ago
Thanks, that explains Reddit. I see the same phenomenon on other smaller forums, too, though. DuckDuckGo always feels like it has a smaller database than Google, which isn't really a surprise.
- skydhash 8mo ago
  I mostly use a web engine (DDG) to find web sites these days, not content. Then I use the site's search instead or just browse the navigation tree. Make everything simpler. I much prefer to use scholar.google.com or npmjs.com for research. The URL is already in my history/bookmarks and the scoped query is more useful than the generic websearch.
thayne 8mo ago
That sounds like some excellent fodder for an anti-trust suit if you ask me.
- alex1138 8mo ago
  It does. Reddit has defined what truth is. Banning r/nonewnormal is merely one part of that
elliotec 8mo ago
Kagi has tons of results from Reddit and they're always high and relevant. I don't know if this means they're doing it even though they're "not allowed to" or what but they definitely get it somehow.
- kyleee 8mo ago
  Kagi is probably paying Google for those results?
  - LeonenTheDK 8mo ago
    When that news first went out, the article[0] I read at the time said that Kagi does purchase some of its indexing from Google. [0] https://www.404media.co/email/4650b997-7cc3-4578-834c-7e663ed3d516/ https://www.404media.co/email/4650b997-7cc3-4578-834c-7e663e...
  - monooso 8mo ago
    I responded to another comment in this thread with the details, but in summary, no. See this previous discussion: https://news.ycombinator.com/item?id=46708678 https://news.ycombinator.com/item?id=46708678
- hoppyhoppy2 8mo ago
  Kagi's search results (at least used to) include many Google search results mixed in with results from other sources. That used to be explained on Kagi's main webpage, but I don't see it there now. (And I don't know who pays whom for what in that type of arrangement.)
  - AlotOfReading 8mo ago
    Kagi uses a third party API that scrapes Google results for their searches. Possibly SerpAPI? Either way, Google doesn't get paid because you can't pay for the kind of search access they want.
- sethops1 8mo ago
  Kagi sources their search results from Google.
  - monooso 8mo ago
    This is false. Kagi had a post discussing this which made the front page of HN about a month ago [1]: > Google does not offer a public search API. The only available path is an ad-syndication bundle with no changes to result presentation - the model Startpage uses. Ad syndication is a non-starter for Kagi’s ad-free subscription model. [1]: https://news.ycombinator.com/item?id=46708678 https://news.ycombinator.com/item?id=46708678
dheera 8mo ago
I'm sure Baidu could safely index Reddit if they wanted to.
RGamma 8mo ago
Holy shit have we come far.

by nomel 8mo ago

> from Reddit threads Google is the only search engine allowed to index Reddit [1]. [1] https://www.lifewire.com/google-reddit-deal-8685766 https://www.lifewire.com/google-reddit-deal-8685766

Aurornis 8mo ago

Thanks, that explains Reddit. I see the same phenomenon on other smaller forums, too, though. DuckDuckGo always feels like it has a smaller database than Google, which isn't really a surprise.

skydhash 8mo ago
I mostly use a web engine (DDG) to find web sites these days, not content. Then I use the site's search instead or just browse the navigation tree. Make everything simpler. I much prefer to use scholar.google.com or npmjs.com for research. The URL is already in my history/bookmarks and the scoped query is more useful than the generic websearch.

thayne 8mo ago

That sounds like some excellent fodder for an anti-trust suit if you ask me.

alex1138 8mo ago
It does. Reddit has defined what truth is. Banning r/nonewnormal is merely one part of that

elliotec 8mo ago

Kagi has tons of results from Reddit and they're always high and relevant. I don't know if this means they're doing it even though they're "not allowed to" or what but they definitely get it somehow.

kyleee 8mo ago
Kagi is probably paying Google for those results?
- LeonenTheDK 8mo ago
  When that news first went out, the article[0] I read at the time said that Kagi does purchase some of its indexing from Google. [0] https://www.404media.co/email/4650b997-7cc3-4578-834c-7e663ed3d516/ https://www.404media.co/email/4650b997-7cc3-4578-834c-7e663e...
- monooso 8mo ago
  I responded to another comment in this thread with the details, but in summary, no. See this previous discussion: https://news.ycombinator.com/item?id=46708678 https://news.ycombinator.com/item?id=46708678
hoppyhoppy2 8mo ago
Kagi's search results (at least used to) include many Google search results mixed in with results from other sources. That used to be explained on Kagi's main webpage, but I don't see it there now. (And I don't know who pays whom for what in that type of arrangement.)
- AlotOfReading 8mo ago
  Kagi uses a third party API that scrapes Google results for their searches. Possibly SerpAPI? Either way, Google doesn't get paid because you can't pay for the kind of search access they want.
sethops1 8mo ago
Kagi sources their search results from Google.
- monooso 8mo ago
  This is false. Kagi had a post discussing this which made the front page of HN about a month ago [1]: > Google does not offer a public search API. The only available path is an ad-syndication bundle with no changes to result presentation - the model Startpage uses. Ad syndication is a non-starter for Kagi’s ad-free subscription model. [1]: https://news.ycombinator.com/item?id=46708678 https://news.ycombinator.com/item?id=46708678

dheera 8mo ago

I'm sure Baidu could safely index Reddit if they wanted to.

RGamma 8mo ago

Holy shit have we come far.