4 ms·
Scraping Amazon might be illegal, or at least not specifically allowed, otherwise there would have been an API. However reading 'robots.txt' I guess that their
by atmosx 12y ago
Scraping Amazon might be illegal, or at least not specifically allowed, otherwise there would have been an API. However reading 'robots.txt' I guess that their policy is 'open to scrapping' by search engines. Because what google spiders do to virtually everyone, can be considered web scrapping.
[1] http://www.amazon.com/robots.txt http://www.amazon.com/robots.txt