3 ms·
Sadly, I do not have that experience. I know what everybody knows. I guess it is extremely hard. Software-Infrastructure wise it is possibly one of the most cha
by afsina 14y ago
Sadly, I do not have that experience. I know what everybody knows. I guess it is extremely hard. Software-Infrastructure wise it is possibly one of the most challenging product. Competition is stiff, so going global is almost not a choice. You either go vertical, or local-national. Yes you can use available open source software (Nutch, Hadoop, Solr etc) but they can make you go up to a point. If you want to go real big, then you may need to write your own crawlers, indexing - ranking, clustering, duplicate-spam detection algorithms, distributed file systems, ad systems etc. Making a search engine of course is not enough, you need to add a gazillion service to make it attractive. Many of those services are also requires cutting edge technology (translation etc.). And it burns money 7/24.
I understand why people values DuckDuckGo for privacy reasons. I respect that. But sure devleoping it, is far far easier than developing a real search engine.
- greglindahl 14y agoIf you're interested in what it takes to write a search engine with its own multi-billion-webpage crawl/index, check out the blog series starting with http://highscalability.com/blog/2012/4/25/the-anatomy-of-search-technology-blekkos-nosql-database.html http://highscalability.com/blog/2012/4/25/the-anatomy-of-sea...
- gummydude 14y agoAnd when they said they're moving towards knowledge graph like instant results, latency not gonna be pretty, lots of small fetchings after general queries from core search servers.