3 ms·
Hmm, I'll have to investigate that, any tips for libraries to use for validation that tie well into scrapy? What do you do if you discover that your parsing lo
by kami8845 11y ago
Hmm, I'll have to investigate that, any tips for libraries to use for validation that tie well into scrapy?
What do you do if you discover that your parsing logic needs to be changed after you've scraped a few thousand items? Re-run your spiders on the URLs that raised errors?
- stummjr 11y agoSpider Contracts can help you: http://doc.scrapy.org/en/latest/topics/contracts.html http://doc.scrapy.org/en/latest/topics/contracts.html