4 ms·
Time records don't have to be scraped from the site or retrieved from APIs (what APIs btw? Isn't it all scraping?). When the search engine crawls a new page, or
by thinkloop 9y ago
Time records don't have to be scraped from the site or retrieved from APIs (what APIs btw? Isn't it all scraping?). When the search engine crawls a new page, or notices an update to a previously crawled page, as long as you're crawling thoroughly, those dates can be pretty accurate.
- JasonSage 9y agoDuckDuckGo does not do all scraping itself, I think. This is why there's the issue of "other sources" not providing dates.
- sitharus 9y agoDuckDuckGo doesn't just run a web crawler, it aggregates from many sources including Bing and area specific APIs. Not all of these sources supply date information so you'd end up with some results having it and others not. Information is here: https://duck.co/help/results/sources https://duck.co/help/results/sources
- thinkloop 9y agoAny idea why search engines would allow API access for a competing search engine? Usually APIs come with a caveat that you can't use them to rebuild the same service. Also for ddg's level of traffic, it seems like special deals would need to made, not just a general access key.
- dredmorbius 9y agoWeakening Google's monopoly.