4 ms·

The search engine basically owns an index of all websites. I assume, that crawling websites and storing an index is a solved problem. Not trivial on that scale

The search engine basically owns an index of all websites. I assume, that crawling websites and storing an index is a solved problem. Not trivial on that scale and expensive. But conceptually solved. What google does really well are two things: 1) It ranks those pages according to the well-known PageRank algorithm, although the details are kept secret. They also learned how to punish sites, which try to mislead the algorithm (Duplicate content, excessive SEO, link farms, etc) 2) What they also do well is returning search results in a very short time, based on a relevancy of your search term combined with the precalculated ranking. It also looks at your history in this step and the data it tracks about you. My preferred search engine would open up both bullet points, giving you the option how pages are ranked and how relevancy is determined. For example: Rank personal blog posts higher or rank news higher or only use an inbound link count. Or you could get fancy and crowd source ranking (similar to hacker news). You could also configure search term relevancy. For example: did you ever search google for something specific and it automatically adjusted your term to something similar, more popular? You could open up all those decisions and give the user options. Google on the other hand keeps everything secret and uses your personal data in unknown ways. I would love to have a search engine which is configurable. Yeah, not an easy thing to do. But it would be awesome.

foota 7y ago
While you may be able to open up some switches, I don't think you can build a reasonable user experience on something like that, since anything that allows you significant control over how the results are chosen (beyond the simple control you've mentioned, like prefer certain types of results) will be both too complex for anything other than like distributed common sets of options (like a plugin that comes with presets) and/or have so many knobs as to make tuning for good results infeasible. I would argue Google already supports some of what you want, like allowing you to search just for news, although additional filters based on something like the niche you're looking for or the type of content don't hurt.

foota 7y ago

While you may be able to open up some switches, I don't think you can build a reasonable user experience on something like that, since anything that allows you significant control over how the results are chosen (beyond the simple control you've mentioned, like prefer certain types of results) will be both too complex for anything other than like distributed common sets of options (like a plugin that comes with presets) and/or have so many knobs as to make tuning for good results infeasible. I would argue Google already supports some of what you want, like allowing you to search just for news, although additional filters based on something like the niche you're looking for or the type of content don't hurt.