4 ms·
If your objective is to write a crawler that parses pricing information from a third party web site, then write the crawler. Relying on the third party to prov
by jijji 11y ago
If your objective is to write a crawler that parses pricing information from a third party web site, then write the crawler. Relying on the third party to provide a consistent API that can be remotely communicated with is setting yourself up for failure. These companies can quickly ban you from their platform when using their API. At least by writing the crawler yourself, you're making it much more difficult for them to do this. Another lesson is that building a business model around the availability of a third party is also a recipe for failure.
- djsumdog 11y agoI don't think that would have mattered here. They were generating revenue from the Affiliate API. Even if they made a scraper and used that for price data instead, they're still violating the TOS for the Affiliate program. It wouldn't matter how they go the data.
- johnward 11y agoI don't remember, nor can I find, any reference to not using historical pricing data with amazon associates: https://affiliate-program.amazon.com/gp/associates/agreement/ https://affiliate-program.amazon.com/gp/associates/agreement... The specific section on pricing in the api doc (https://affiliate-program.amazon.com/gp/advertising/api/detail/agreement.html/ref=amb_link_353005802_15?pf_rd_m=ATVPDKIKX0DER&pf_rd_s=assoc-center-1&pf_rd_r=&pf_rd_t=501&pf_rd_p=&pf_rd_i=assoc_operating https://affiliate-program.amazon.com/gp/advertising/api/deta...) seems to indicate historical pricing is allowed if you display the date with it: "(o) You will include a date/time stamp adjacent to your display of pricing or availability information on your application if you obtain Product Advertising Content from a Data Feed, or if you call the Product Advertising API or refresh the Product Advertising Content displayed on your application less frequently than hourly. However, during the same day on which you requested and refreshed the pricing and availability information displayed on your application, you may omit the date portion of the stamp. Examples of acceptable messaging include: Amazon.com Price: $32.77 (as of 01/07/2008 14:11 PST - Details) Amazon.com Price: $32.77 (as of 14:11 PST - More info) Additionally, you must either include the following disclaimer adjacent to the pricing or availability information or provide it via a hyperlink, popup box, scripted popup, or other similar method: "Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on [relevant Amazon Site(s), as applicable] at the time of purchase will apply to the purchase of this product." In the above examples, "Details" and "More info" would provide a method for the end user to read the disclaimer." Update: I'm thinking it's probably a violation of this: "(n) You will not store or cache Product Advertising Content consisting of an image, but you may store a link to Product Advertising Content consisting of an image for up to 24 hours. You may store other Product Advertising Content that does not consist of images for caching purposes for up to 24 hours, but if you do so you must immediately thereafter refresh and re-display the Product Advertising Content by making a call to the Product Advertising API or retrieving a new Data Feed and refreshing the Product Advertising Content on your application immediately thereafter."
- johnward 11y agoNot that you actually have to break TOS. Amazon can shut down your access for no reason at all.
- pcarmichael 11y agoIt's in the section (n) just before that, regarding not storing API content longer than 24 hours.
- josho 11y ago> building a business model around the availability of a third party is a recipe for failure. Rather, I think it's that a business model that is not complimentary to the third party is the problem. E.g. Amazon doesn't want historical price tracking for as it could lead to folks discovering price manipulations and a decrease in margins. So, that's the issue--the business was not a compliment. Let's imagine a startup that was generating new sales from a long tail market segment too small for Amazon to target directly, then that's a good business model as it compliments Amazon and leads to new sales they wouldn't otherwise see. TLDR if you rely on a partner, make sure your business helps their business.