3 ms·
According to the article, that would not have helped; feedfetcher is meant to be manually triggered and thus does not obey robots.txt
by eli 15y ago
According to the article, that would not have helped; feedfetcher is meant to be manually triggered and thus does not obey robots.txt
- tripzilch 15y agoFor certain definitions of "manually" :) It's manually triggered to start downloading resources every hour regardless of whether someone needs them. In that sense, any web spider is "manually triggered" as well ;-)