4 ms·
In a discussion about this at work a few months back, someone suggested to me that part of Google's motivation might be to increase the cost of entry for search
by threedaymonk 14y ago
In a discussion about this at work a few months back, someone suggested to me that part of Google's motivation might be to increase the cost of entry for search competitors. No longer can you just scrape a page and index it: not one of the 22 KB served at that URL actually contains the text of the page. To scrape one of those awful pages, you need to JavaScript interpreter and a DOM - a headless browser, really. Unless you're Google: they can just index the back-end directly
I'm not entirely convinced, but I've heard worse conspiracy theories that turned out to be true.
- uptown 14y agoGetting a headless browser up and running really isn't that huge of a hurdle if you're capable enough to build a search engine whose intent is to be competitive with Google.
- threedaymonk 14y agoIt's not technically hard, no. However, there's an overhead in running a headless browser, of memory, CPU, time, and bandwidth, so it might cost substantially more. You don't need a browser for everything, but working out which pages need a headless browser and which you can get away with scraping sounds like an interesting problem.
- itsadok 14y agoYou can read the entire blog from the RSS feed. Can't say that about Google+, though. http://feeds.feedburner.com/OfficialGmailBlog http://feeds.feedburner.com/OfficialGmailBlog