It's almost impossible to build a decent web search engine from scratch today (i.e., build your own index, fight SEO spams, tweak search result relevance...). T

It's almost impossible to build a decent web search engine from scratch today (i.e., build your own index, fight SEO spams, tweak search result relevance...). The web is already so big and so complex. Otherwise Google won't need to hire so many people to work on search alone. If you didn't start at the very early stage of tiny web (e.g., Google in 1996 as a research project) and grew with the web over the past 20+ years, or you don't have super deep pocket (e.g., Microsoft Bing in mid 2000s), then it's almost impossible to build a decent web search engine within a few years. It's possible to build vertical search engines on far smaller scale, far less complex, far less lucrative things that Google/Microsoft has little interest today (e.g., recipes [2], podcasts [3], gifs [4]...) It's also possible to come up with a different discovery mechanism for web (or a small portion of web), other than a traditional complete web search engine. Essentially you don't cross moat to attack a huge castle (e.g., Google). Instead, you bypass the castle [1], as it becomes irrelevant. [1] https://twitter.com/benedictevans/status/1038538688232226817?lang=en https://twitter.com/benedictevans/status/1038538688232226817... [2] https://www.yummly.com/ https://www.yummly.com/ [3] https://www.listennotes.com/ https://www.listennotes.com/ [4] https://giphy.com/ https://giphy.com/