4 ms·
@Gabriel: How long do you think this will be a sustainable approach to powering Web search? What I mean is, after a certain amount of traffic will you be able t
by coderdude 16y ago
@Gabriel: How long do you think this will be a sustainable approach to powering Web search? What I mean is, after a certain amount of traffic will you be able to continue using all these external APIs or do you expect that in the future you will have to augment some or all of this data with something in-house like moving to full-scale Web crawls?
- epi0Bauqu 16y agoIt is quite sustainable. These APIs are central to the business models of the companies and they should generally scale up as needed. For example, SeatGeek already powers WSJ and Yahoo Sports.
- coderdude 16y agoGood to know. What are your plans for crawling in the future? Do you think that you will eventually move to crawling as a primary source of result data, and if so, is that time drawing near or do you plan on continuing along the current path for some time?
- epi0Bauqu 16y agoWe never stopped crawling; we just scaled it back for specific purposes, mainly now for spam detection and zero-click info. For the foreseeable future this status quo works well because we can concentrate on our value-ads.