3 ms·
The author has just decided to roll their own half baked script for some reason, there are plenty of libraries to help. Python is good for scraping, I released
by orf 11y ago
The author has just decided to roll their own half baked script for some reason, there are plenty of libraries to help. Python is good for scraping, I released my own library recently[1] that I think hits a sweet spot but there are many others.
Python and Ruby have their places, I wouldnt say one is clearly better than the other.
1. http://tomforb.es/scraping-websites-with-cyborg http://tomforb.es/scraping-websites-with-cyborg
- rangeva 11y agoThe script is intended to be a copy/paste script, with no imports and just "Fill in the blanks" parameters. There A LOT of other options that are much better, but require more setup: http://scrapy.org/ http://scrapy.org/ https://pypi.python.org/pypi/spider.py https://pypi.python.org/pypi/spider.py
- aerioux 11y agoupvote - python has ridiculously good (sometimes slowish) scraping libraries, unfortunately the author has written a python wrapper over a regex which completely throws any half-sane crawling out the window. The except: and continues is enough to cause a heartattack