3 ms·
Signing up wasn't that hard - they just want minimum sales per 90 days. I still don't have API access though; figured it was worth just trying this and seeing w
by generalizations 3y ago
Signing up wasn't that hard - they just want minimum sales per 90 days. I still don't have API access though; figured it was worth just trying this and seeing what happened. It's super unclear what leverage Amazon tries to exert; it might be that they just refuse API access? Guess I'll find out when/if I try to withdraw my winnings.
- leros 3y agoGood to know. Thanks for the info. I'm thinking about building my own price comparison site in another niche. How are you scraping the data right now?
- generalizations 3y agoAn unholy combination of bash, python and TCL scripts running in async loops on an old computer under my desk :D TCL is actually a great scripting language - it handles text strings as well as bash (and this is all just text), it does file I/O really easily, and it has nested arrays that bash doesn't do. For my purposes, it's a lot cleaner to work with than python. For the scraping itself - I tried wget with useragent strings, and quickly hit 503 blocked errors. After that I gave it the whole nine yards, and threw every trick I could come up with at the problem (considering my last project involved netbooting xen, this was completely outside my field). I honestly would rather not say how I did it though. If anyone from Amazon's bot detection is here, I don't want to give them hints. Edit: I'm collecting similar projects in the sidebar of my site, it'd be fun to add yours when you publish it.