4 ms·

What you're missing is the fact that larch search engine companies have no profit motive to do this. They get the vast amount of their money from commercial ad

What you're missing is the fact that larch search engine companies have no profit motive to do this. They get the vast amount of their money from commercial ad sales -- ads for commercial results. Why would they purposefully castrate their highest paying customers? And if you think the answer is to create and host a new, independent engine, you will be hard pressed to develop a good one, and to find the money to keep it running.

swayvil 7y ago
How about a layer over Google? Take Google's search results and filter them. Any obvious barriers to that?
- tyingq 7y ago
  They are pretty aggressive about detecting that and throwing up captchas.
  - swayvil 7y ago
    see user smt88's idea above.
- jasode 7y ago
  >Any obvious barriers to that? Rate limits.[0] One can't really build a robust and comprehensive (server-side) search engine on top of Google's search API. In other words, Google Inc isn't going to provide an API so powerful that one can build an "alternative to Google" with it. On the other hand, if you're talking about a "client-side" filter instead of a new search engine, (e.g. a client-side webscraper that flips through the Google results pages), you'd run into multiple problems. E.g. random CAPTCHAS screwing up the automation script, and the difficulty of creating a "rules engine" (e.g. machine learning) on your local computer that recognizes blog articles with a commercial slant. [0] https://www.google.com/search?q=google+search+api+rate+limit https://www.google.com/search?q=google+search+api+rate+limit
  - swayvil 7y ago
    see user smt88's idea above.
swayvil 7y ago
user smt88 offered a fabulous idea : >Wikipedia is basically an information search engine without commerce or social media. The problem is that your results page is the article itself, and you need to scroll to the bottom to get to an external site. >It's theoretically possible to change the UI or analyze Wikipedia to make a pretty solid search engine powered by millions of person-years of curation.
- jasode 7y ago
  >user smt88 offered a fabulous idea : >Wikipedia is basically an information search engine without commerce or social media. The problem is that your results page is the article itself, and you need to scroll to the bottom to get to an external site. Using wikipedia as a data source for a "search engine" does not fulfill gp's (jacquesm) request of "One nice solution would be a search engine that throws out anything related to commerce or linking to commerce." For example, go to the wikipedia page for the film "Groundhog Day" and look at the external links: https://en.wikipedia.org/wiki/Groundhog_Day#External_links https://en.wikipedia.org/wiki/Groundhog_Day#External_links - one of the external links points to IMDB.com -- a commercial website owned by Amazon. - another link points to Punxsatawney Groundhog Club website that's advertising a $30 beer tasting event (Hogtoberfest). - another link points to a souvenir shop To continue the example of commercial links, a lot of math articles on wikipedia point to MathWorld which has advertising for the commercial Wolfram Mathematica software package. IMO, I don't believe a search engine based on wikipedia's list of external sites (even with ignoring the commercial links) is going to be useful for the mainstream audience. I probably do 50+ searches on google every single day and maybe 1% might be answerable from a "wikipedia search". For example, I needed a DIY answer to disassemble a Moen faucet and Wikipedia has zero articles with those instructions. In contrast, the Google (and Youtube) search results has the information I was looking for even though it has the unwanted ads and "content marketing" blogs from plumbing brands.

swayvil 7y ago

How about a layer over Google? Take Google's search results and filter them. Any obvious barriers to that?

tyingq 7y ago
They are pretty aggressive about detecting that and throwing up captchas.
- swayvil 7y ago
  see user smt88's idea above.
jasode 7y ago
>Any obvious barriers to that? Rate limits.[0] One can't really build a robust and comprehensive (server-side) search engine on top of Google's search API. In other words, Google Inc isn't going to provide an API so powerful that one can build an "alternative to Google" with it. On the other hand, if you're talking about a "client-side" filter instead of a new search engine, (e.g. a client-side webscraper that flips through the Google results pages), you'd run into multiple problems. E.g. random CAPTCHAS screwing up the automation script, and the difficulty of creating a "rules engine" (e.g. machine learning) on your local computer that recognizes blog articles with a commercial slant. [0] https://www.google.com/search?q=google+search+api+rate+limit https://www.google.com/search?q=google+search+api+rate+limit
- swayvil 7y ago
  see user smt88's idea above.

swayvil 7y ago

user smt88 offered a fabulous idea : >Wikipedia is basically an information search engine without commerce or social media. The problem is that your results page is the article itself, and you need to scroll to the bottom to get to an external site. >It's theoretically possible to change the UI or analyze Wikipedia to make a pretty solid search engine powered by millions of person-years of curation.

jasode 7y ago
>user smt88 offered a fabulous idea : >Wikipedia is basically an information search engine without commerce or social media. The problem is that your results page is the article itself, and you need to scroll to the bottom to get to an external site. Using wikipedia as a data source for a "search engine" does not fulfill gp's (jacquesm) request of "One nice solution would be a search engine that throws out anything related to commerce or linking to commerce." For example, go to the wikipedia page for the film "Groundhog Day" and look at the external links: https://en.wikipedia.org/wiki/Groundhog_Day#External_links https://en.wikipedia.org/wiki/Groundhog_Day#External_links - one of the external links points to IMDB.com -- a commercial website owned by Amazon. - another link points to Punxsatawney Groundhog Club website that's advertising a $30 beer tasting event (Hogtoberfest). - another link points to a souvenir shop To continue the example of commercial links, a lot of math articles on wikipedia point to MathWorld which has advertising for the commercial Wolfram Mathematica software package. IMO, I don't believe a search engine based on wikipedia's list of external sites (even with ignoring the commercial links) is going to be useful for the mainstream audience. I probably do 50+ searches on google every single day and maybe 1% might be answerable from a "wikipedia search". For example, I needed a DIY answer to disassemble a Moen faucet and Wikipedia has zero articles with those instructions. In contrast, the Google (and Youtube) search results has the information I was looking for even though it has the unwanted ads and "content marketing" blogs from plumbing brands.