3 ms·
It's analogous to predatory pricing. Google has all the necessary tech to go back to doing good searches. They can easily pivot back in a few months. A new sta
by wmil 5y ago
It's analogous to predatory pricing. Google has all the necessary tech to go back to doing good searches. They can easily pivot back in a few months.
A new startup would need to dump loads of money into servers and building their own tech.
So there's just no way to recoup the cost of building a competitor.
- systemvoltage 5y agoYou might be surprised how much compute it takes. Not the same thing but, entire SO website runs on like 2 door sized racks. Search engine might be a different thing but if you have funding to get started, hardware costs aren’t going to be impossibly huge. Most is labor (engineering). I’m curious, how much compute power it takes to index the whole web? I presume queries are super fast.
- alangibson 5y agoIMHO, indexing the whole web is an antifeature for technical and research topics. But there are projects like CommonCrawl that crawl a good portion of the web and give it away free.
- estebarb 5y agoThe monthly snapshots of CommonCrawl contains around 64k files of HTML, metadata and text only. The text only and metadata are around 250MB each, so in total the corpus would be around 16TB of text and 16TB of metadata. I don't know how much it costs to them, but when I used my own crawler I was able to download 500k webpages in around 4 hours (using the $80/month basic DigitalOcean server... let's say a total of $1/1M web pages, just the crawling process).