3 ms·

>I really wish a search engine would empower me to provide a URL ban list Brave discussed implementing similar concept called Goggles: https://brave.com/wp-co

>I really wish a search engine would empower me to provide a URL ban list Brave discussed implementing similar concept called Goggles: https://brave.com/wp-content/uploads/2021/03/goggles.pdf https://brave.com/wp-content/uploads/2021/03/goggles.pdf Kagi Search already has customizable 'prefer' and 'mute' lists for domains, as well as customizable 'lenses' which are similar to goggles concept from Brave. (disclaimer: founder at Kagi) https://kagi.com https://kagi.com

hbn 5y ago
I'm not even sure how much manually blocking domains would help remedy the problem with Google results, which is that so much of the top hits are copy-pasted blog spam with very little information, scrambled together (very possibly by a non-human) purely to game SEO. It seems like it's gotten even worse just in the past few months. I'll try to Google something common, like "can you feed X to dogs" and all the results I find are these giant "articles" that aren't even about that specific thing. Instead it's a giant wall of text with commonly-Googled questions, and if you ctrl+F to the section you were looking for, the answer is usually horrible, and why should you be trusting information from this website anyway? Then you go back and click through other results and find an entirely different website with all the exact same text on it. Top recipe results are also all copy-pasted SEO spam surrounded by a wall of text about the history of the recipe, and how the author is a "country mom" (definitely not a man at a content farm in India), hiding the recipe deep within. They must be pumping out all these fake dedicated websites at such a high rate that blocking domains won't get you anywhere.
- SllX 5y ago
  Sure it would help. Adblockers have had custom filter lists for years. If all you need to do is upload a text file every once in a while, that would go a long way to improving search results in general.
  - hbn 5y ago
    I didn't click into the paper, but I see now it also mentions crowdsourced filters. That would be much more effective than everyone creating their own blocklists
- freediver 5y ago
  What we do (at Kagi Search) to address this problem is use our own index which contains only non-commercial results from the web as well as forum discussions. Thinking behind is that the quality of the content is in inverse proportion to the number of ads/trackers/affiliate links on the site. Our index is still in infancy but the example of it working for the query 'best laptop' can be seen in this screenshot: https://imgur.com/ypyOilV https://imgur.com/ypyOilV
  - dantyti 5y ago
    afaik, seo spammers are quite sophisticated in masking affiliate links and tracking codes with redirect chains and other techniques. Their general assumption is that google does exactly what you expect to do and downranks affiliate spam. What are your thoughts on reputable news websites that have tons of ads and trackers because they have yet to figure out any other viable way to survive?
    - freediver 5y ago
      I think that 'reputable' and 'tons of ads and trackers' do not belong in the same sentence. Ad-supported business models incentivize the creation of large quantities of content, because you need a lot of pageviews to earn a little bit of money with ads (since most people either ignore or block ads, if they manage to load the page to see them at all). High quality journalism should have value that we are ready to pay for, like we did for hundreds of years. This concept is called the "newspaper". You are expected to pay for high quality baker or a tailor, book, movie or music, why not for getting information that "only" has the power to shape societies? If I see information next to an ad, I personally tend not take it too seriously. What is missing right now from a technology standpoint is an easy way to manage subscriptions, built into my browser as default. Expecting the user to create and manage a separate account/billing identity for every publisher is what is preventing this model to take off (IMO).
      - lanterk 5y ago
        And people already complain about paywalls. And even make tools to skip past the paywalls.

hbn 5y ago

I'm not even sure how much manually blocking domains would help remedy the problem with Google results, which is that so much of the top hits are copy-pasted blog spam with very little information, scrambled together (very possibly by a non-human) purely to game SEO. It seems like it's gotten even worse just in the past few months. I'll try to Google something common, like "can you feed X to dogs" and all the results I find are these giant "articles" that aren't even about that specific thing. Instead it's a giant wall of text with commonly-Googled questions, and if you ctrl+F to the section you were looking for, the answer is usually horrible, and why should you be trusting information from this website anyway? Then you go back and click through other results and find an entirely different website with all the exact same text on it. Top recipe results are also all copy-pasted SEO spam surrounded by a wall of text about the history of the recipe, and how the author is a "country mom" (definitely not a man at a content farm in India), hiding the recipe deep within. They must be pumping out all these fake dedicated websites at such a high rate that blocking domains won't get you anywhere.

SllX 5y ago
Sure it would help. Adblockers have had custom filter lists for years. If all you need to do is upload a text file every once in a while, that would go a long way to improving search results in general.
- hbn 5y ago
  I didn't click into the paper, but I see now it also mentions crowdsourced filters. That would be much more effective than everyone creating their own blocklists
freediver 5y ago
What we do (at Kagi Search) to address this problem is use our own index which contains only non-commercial results from the web as well as forum discussions. Thinking behind is that the quality of the content is in inverse proportion to the number of ads/trackers/affiliate links on the site. Our index is still in infancy but the example of it working for the query 'best laptop' can be seen in this screenshot: https://imgur.com/ypyOilV https://imgur.com/ypyOilV
- dantyti 5y ago
  afaik, seo spammers are quite sophisticated in masking affiliate links and tracking codes with redirect chains and other techniques. Their general assumption is that google does exactly what you expect to do and downranks affiliate spam. What are your thoughts on reputable news websites that have tons of ads and trackers because they have yet to figure out any other viable way to survive?
  - freediver 5y ago
    I think that 'reputable' and 'tons of ads and trackers' do not belong in the same sentence. Ad-supported business models incentivize the creation of large quantities of content, because you need a lot of pageviews to earn a little bit of money with ads (since most people either ignore or block ads, if they manage to load the page to see them at all). High quality journalism should have value that we are ready to pay for, like we did for hundreds of years. This concept is called the "newspaper". You are expected to pay for high quality baker or a tailor, book, movie or music, why not for getting information that "only" has the power to shape societies? If I see information next to an ad, I personally tend not take it too seriously. What is missing right now from a technology standpoint is an easy way to manage subscriptions, built into my browser as default. Expecting the user to create and manage a separate account/billing identity for every publisher is what is preventing this model to take off (IMO).
    - lanterk 5y ago
      And people already complain about paywalls. And even make tools to skip past the paywalls.