2 ms·
Let me just ask the $64.000 question. Why did they ever bothered with the API and not just crawl Amazon for whatever products they needed?
by elorant 11y ago
Let me just ask the $64.000 question. Why did they ever bothered with the API and not just crawl Amazon for whatever products they needed?
- lazyjones 11y agoCrawling Amazon is no longer feasible if you want up-to-date prices. There's 10s of millions of pages to crawl, it takes too long. I believe they also added captchas against excessive crawling (perhaps via Tor only).
- sangnoir 11y agoIt wouldn't make a difference: it is their affiliate account was that was closed - which is how they earned money. Affiliate account = getting paid "kickbacks" by Amazon for customer referalls from PZ's site that result in purchases.
- cookiecaper 11y agoIt wouldn't matter. If Amazon didn't want them accessing their servers, they could file suit under the CFAA. They could also allege copyright infringement, since judges have ruled that the temporary copy of a page stored in RAM from which factual non-copyrightable data is extracted (as would be the case on an Amazon product page) constitutes an infringing copy, and trademark infringement and dilution. The existing body of law is very hostile to scrapers. Server access is treated like access to land. The "owner" can kick you off whenever they want.