4 ms·

Can the script on this then be flipped? Build a search engine, clearly smaller in scope and commercial utility, that if a site links to a payment or ad network,

by Fordec 2y ago

Can the script on this then be flipped? Build a search engine, clearly smaller in scope and commercial utility, that if a site links to a payment or ad network, de-rank it heavily. Then the end result should be in theory, filled with what one would consider the "old" internet, primarily blogs and sites not trying to sell you things or abuse your data. None of the large companies would do it, but that would be the point.

JumpCrisscross 2y ago
Introducing: Kagi.
- sph 2y ago
  That's not what Kagi does. Nor does its "Small Web" search mode, as it only searches blogs that have been manually added to a specific GitHub repo (so for most part is a collection of US tech blogs - not very diverse at all) No VC-backed or commercial search engine would do what OP is talking about. But I can see a use for a niche search engine that ranks websites inversely proportional to the number of trackers and ad networks they depend on. Heck, I would pay for that, but I'm a nerd. Maybe marginalia.nu would like this idea.
  - JumpCrisscross 2y ago
    > not what Kagi does Kagi maintains a "non-commercial index (Teclis) and non-commercial news index (TinyGem)" [1]. They also "prioritize non-commercial sources," implicitly downranking monetized sites. (Also, you can manually downrank problem domains [2].) My devices have randomly switched back to Google or DDG from time to time. The first thing I did was check my ad blocker was working--I was simply stunned by the amount of blogspam puke. > No VC-backed or commercial search engine would do what OP is talking about No ad based, i.e. free, engine can, not sustainably. A paid one obviously can and does. [1] https://help.kagi.com/kagi/search-details/search-quality.html https://help.kagi.com/kagi/search-details/search-quality.htm... [2] https://help.kagi.com/kagi/features/website-info-personalized-results.html https://help.kagi.com/kagi/features/website-info-personalize...
  - fintler 2y ago
    https://kagi.com/smallweb https://kagi.com/smallweb
  - freediver 2y ago
    > That's not what Kagi does. We do actually. We penalize the sites with a lots of ads/trackers on them in our results and boost non-monetized pages. It is one of the main reasons Kagi reults have a specific 'flavor' to them. (Kagi CEO here)
    - sph 2y ago
      You surely have huge exceptions to this rule, or major silos like Reddit, Twitter, StackOverflow, Youtube, Facebook etc. would rank towards the bottom. Which the average Joe wouldn't like, hence my comment that no commercial search engine would implement this feature correctly, on purpose.
      - freediver 2y ago
        Not really exceptions (all treated equally) just there is nuance in the algorithm. Having some ads doesn't mean you go straight to the bottom. And when searching for youtube you clearly want youtube.com - so nuance applied but that is the principle driving it.
- baxtr 2y ago
  What’s the reason that search engine would ask for a login? (Like Kagi does)
  - deleted 2y ago
    [deleted]
  - TeMPOraL 2y ago
    Because it's a paid service? That's the entire point. And that also enables tons of user-centric features they talked about, starting with the earliest ones and my favorites: being able to uprank and downrank domains (like, "ban pinterest", "pin Wikipedia", "downrank w3schools", "uprank cppreference", etc.), and adding rewrite rules to results (like "reddit.com" -> "old.reddit.com"). Both of these are personalizations tied to your account, so they're active on any device as long as you're logged in. They've added more cool stuff since, but these two alone were what has kept me a paying customer for the past years.
    - baxtr 2y ago
      Ok the paid part makes of course sense. However I still find it a bit creepy to know that they know all about my searches and even up- and down votes. Google at least can be used in incognito mode.
      - evoke4908 2y ago
        Google knows what you search, even in "incognito" mode, and even when you're logged out. They correlate your IP address with your search profile and use everything you search for in your ad profile. Kagi does not track search history, period. There's no history attached to your account even if you wanted it. The login is purely for authentication.
Ferret7446 2y ago
Those sites don't exist any more. Not literally of course, but effectively. There's probably only a couple tens of thousand if I had to guess (and many of them abandoned blogs with a couple dozen pages of no note, hosted by some server that has not yet been unplugged because it's physically lost). Also, good luck trying to find them (either electronically or physically). (And I say that as someone who owns such a site.) Case in point, I wanted to link to http://bash.org/?5273 http://bash.org/?5273, but bash.org no longer exists.
farseer 2y ago
You can use Wikipedia's built in search for that I guess.

by Fordec 2y ago

JumpCrisscross 2y ago

Introducing: Kagi.

sph 2y ago
That's not what Kagi does. Nor does its "Small Web" search mode, as it only searches blogs that have been manually added to a specific GitHub repo (so for most part is a collection of US tech blogs - not very diverse at all) No VC-backed or commercial search engine would do what OP is talking about. But I can see a use for a niche search engine that ranks websites inversely proportional to the number of trackers and ad networks they depend on. Heck, I would pay for that, but I'm a nerd. Maybe marginalia.nu would like this idea.
- JumpCrisscross 2y ago
  > not what Kagi does Kagi maintains a "non-commercial index (Teclis) and non-commercial news index (TinyGem)" [1]. They also "prioritize non-commercial sources," implicitly downranking monetized sites. (Also, you can manually downrank problem domains [2].) My devices have randomly switched back to Google or DDG from time to time. The first thing I did was check my ad blocker was working--I was simply stunned by the amount of blogspam puke. > No VC-backed or commercial search engine would do what OP is talking about No ad based, i.e. free, engine can, not sustainably. A paid one obviously can and does. [1] https://help.kagi.com/kagi/search-details/search-quality.html https://help.kagi.com/kagi/search-details/search-quality.htm... [2] https://help.kagi.com/kagi/features/website-info-personalized-results.html https://help.kagi.com/kagi/features/website-info-personalize...
- fintler 2y ago
  https://kagi.com/smallweb https://kagi.com/smallweb
- freediver 2y ago
  > That's not what Kagi does. We do actually. We penalize the sites with a lots of ads/trackers on them in our results and boost non-monetized pages. It is one of the main reasons Kagi reults have a specific 'flavor' to them. (Kagi CEO here)
  - sph 2y ago
    You surely have huge exceptions to this rule, or major silos like Reddit, Twitter, StackOverflow, Youtube, Facebook etc. would rank towards the bottom. Which the average Joe wouldn't like, hence my comment that no commercial search engine would implement this feature correctly, on purpose.
    - freediver 2y ago
      Not really exceptions (all treated equally) just there is nuance in the algorithm. Having some ads doesn't mean you go straight to the bottom. And when searching for youtube you clearly want youtube.com - so nuance applied but that is the principle driving it.
baxtr 2y ago
What’s the reason that search engine would ask for a login? (Like Kagi does)
- deleted 2y ago
  [deleted]
- TeMPOraL 2y ago
  Because it's a paid service? That's the entire point. And that also enables tons of user-centric features they talked about, starting with the earliest ones and my favorites: being able to uprank and downrank domains (like, "ban pinterest", "pin Wikipedia", "downrank w3schools", "uprank cppreference", etc.), and adding rewrite rules to results (like "reddit.com" -> "old.reddit.com"). Both of these are personalizations tied to your account, so they're active on any device as long as you're logged in. They've added more cool stuff since, but these two alone were what has kept me a paying customer for the past years.
  - baxtr 2y ago
    Ok the paid part makes of course sense. However I still find it a bit creepy to know that they know all about my searches and even up- and down votes. Google at least can be used in incognito mode.
    - evoke4908 2y ago
      Google knows what you search, even in "incognito" mode, and even when you're logged out. They correlate your IP address with your search profile and use everything you search for in your ad profile. Kagi does not track search history, period. There's no history attached to your account even if you wanted it. The login is purely for authentication.

Ferret7446 2y ago

Those sites don't exist any more. Not literally of course, but effectively. There's probably only a couple tens of thousand if I had to guess (and many of them abandoned blogs with a couple dozen pages of no note, hosted by some server that has not yet been unplugged because it's physically lost). Also, good luck trying to find them (either electronically or physically). (And I say that as someone who owns such a site.) Case in point, I wanted to link to http://bash.org/?5273 http://bash.org/?5273, but bash.org no longer exists.

farseer 2y ago

You can use Wikipedia's built in search for that I guess.