4 ms·
beautifulsoup
by bantersaurus 9y ago
beautifulsoup
- oddeyed 9y agoAlso good is RoboBrowser which combines beautifulsoup with Requests to get a nice 'Browser' abstraction. It also has good built-in functionality for filling in forms.
- cjsuk 9y agoUsing this as well with Requests to automate eBay/gumtree/craigslist. Works very well
- djaychela 9y agoAny details on this anywhere, or is it not for public consumption? I'm just getting started in Python and want to do something with Gumtree and eBay as an idea to help me in a different sphere.
- cjsuk 9y agoIt's not really for public consumption because it's embarrassingly badly written :) It's pretty dumb really. Just figured out the search URLs and then parse the list responses. It then stores the auctions/ad IDs it has seen in a tiny redis instance with 60 days' expiry on each ID it inserts. If there are any items it hasn't seen each time it runs, it compiles them in a list and emails them to me via AWS SNS. Runs every 5 minutes from cron on a Raspberry Pi Zero plugged into the back of my XBox 360 as a power supply and my router via a USB/ethernet cable. The main bulk of the work went into the searches to run which are a huge list of typos on things with a high return. I tend to buy, test, then reship them for profit. Not much investment gives a very good return - pays for the food bill every month :)
- djaychela 9y agoThanks for the info - I'm sure mine will be of lower quality when I do write it - hoping to compile real-world info on sold vehicles by scraping info from eBay and Gumtree, but that will take time and more skills than I currently possess. Good to hear someone's made something out of a similar idea, though.
- cjsuk 9y agoSounds like a good idea. Good luck - you can do it! :)