4 ms·
How did you select those disks? Did you just hard code a list of ASINs to query through their API or is this done by scraping Amazon front-end?
by antisthenes 9y ago
How did you select those disks? Did you just hard code a list of ASINs to query through their API or is this done by scraping Amazon front-end?
- synack 9y agoAmazon's product categories are identified as BrowseNodes, each has a unique ID. I found the IDs of all the relevant BrowseNodes, then use the API to query what products are available in each BrowseNode. I then use a ridiculous list of regexes to parse the product names to figure out the capacity and drive type, as Amazon's tags for those fields are really unreliable.
- antisthenes 9y agoI only ask because the HDD list in particular seemed to be quite short. The last time I did this, I scraped the internal hard drive category (from the front-end) and filtered everything out that was smaller than 4TB and still ended up with at least 200 drives. For tags, are you using the "Size" or the <ns2:HardDiskSize Units="TB">4.0</ns2:HardDiskSize> tag? I did a few test queries and found those specific tags to be pretty reliable. If you're open to it, we can discuss it outside of HN to see if there's a better way. disclaimer: I do not have any vested interest in Amazon affiliate programs, I work with their API for my day job and always looking to learn. *edit: are you using the advertising api or the mws product api?