3 ms·

> I don't want Google results! I want different web crawlers ordering the results according to my taste Okay, great, we'll see how you browse and use that to d

by foota 7y ago

> I don't want Google results! I want different web crawlers ordering the results according to my taste Okay, great, we'll see how you browse and use that to determine... > without tracking each and every page impression of me. Give me that and I'll switch in a heartbeat. Uh? How exactly do you want them to do that?

zubspace 7y ago
The search engine basically owns an index of all websites. I assume, that crawling websites and storing an index is a solved problem. Not trivial on that scale and expensive. But conceptually solved. What google does really well are two things: 1) It ranks those pages according to the well-known PageRank algorithm, although the details are kept secret. They also learned how to punish sites, which try to mislead the algorithm (Duplicate content, excessive SEO, link farms, etc) 2) What they also do well is returning search results in a very short time, based on a relevancy of your search term combined with the precalculated ranking. It also looks at your history in this step and the data it tracks about you. My preferred search engine would open up both bullet points, giving you the option how pages are ranked and how relevancy is determined. For example: Rank personal blog posts higher or rank news higher or only use an inbound link count. Or you could get fancy and crowd source ranking (similar to hacker news). You could also configure search term relevancy. For example: did you ever search google for something specific and it automatically adjusted your term to something similar, more popular? You could open up all those decisions and give the user options. Google on the other hand keeps everything secret and uses your personal data in unknown ways. I would love to have a search engine which is configurable. Yeah, not an easy thing to do. But it would be awesome.
- foota 7y ago
  While you may be able to open up some switches, I don't think you can build a reasonable user experience on something like that, since anything that allows you significant control over how the results are chosen (beyond the simple control you've mentioned, like prefer certain types of results) will be both too complex for anything other than like distributed common sets of options (like a plugin that comes with presets) and/or have so many knobs as to make tuning for good results infeasible. I would argue Google already supports some of what you want, like allowing you to search just for news, although additional filters based on something like the niche you're looking for or the type of content don't hurt.
phreeza 7y ago
I guess at least in principle, the search engines could return many more results than actually needed, along with some feature vector per result. Then a model could be trained on the users machine to sort the results according to their preferences

by foota 7y ago

zubspace 7y ago

The search engine basically owns an index of all websites. I assume, that crawling websites and storing an index is a solved problem. Not trivial on that scale and expensive. But conceptually solved. What google does really well are two things: 1) It ranks those pages according to the well-known PageRank algorithm, although the details are kept secret. They also learned how to punish sites, which try to mislead the algorithm (Duplicate content, excessive SEO, link farms, etc) 2) What they also do well is returning search results in a very short time, based on a relevancy of your search term combined with the precalculated ranking. It also looks at your history in this step and the data it tracks about you. My preferred search engine would open up both bullet points, giving you the option how pages are ranked and how relevancy is determined. For example: Rank personal blog posts higher or rank news higher or only use an inbound link count. Or you could get fancy and crowd source ranking (similar to hacker news). You could also configure search term relevancy. For example: did you ever search google for something specific and it automatically adjusted your term to something similar, more popular? You could open up all those decisions and give the user options. Google on the other hand keeps everything secret and uses your personal data in unknown ways. I would love to have a search engine which is configurable. Yeah, not an easy thing to do. But it would be awesome.

foota 7y ago
While you may be able to open up some switches, I don't think you can build a reasonable user experience on something like that, since anything that allows you significant control over how the results are chosen (beyond the simple control you've mentioned, like prefer certain types of results) will be both too complex for anything other than like distributed common sets of options (like a plugin that comes with presets) and/or have so many knobs as to make tuning for good results infeasible. I would argue Google already supports some of what you want, like allowing you to search just for news, although additional filters based on something like the niche you're looking for or the type of content don't hurt.

phreeza 7y ago

I guess at least in principle, the search engines could return many more results than actually needed, along with some feature vector per result. Then a model could be trained on the users machine to sort the results according to their preferences