3 ms·
Just in time. That XPath example is really useful for me. I didn't finished the article yet, but I have one question. Is there ability in Scrapy to append new
by slezyr 9y ago
Just in time. That XPath example is really useful for me.
I didn't finished the article yet, but I have one question. Is there ability in Scrapy to append new data after initial scrapping?
For example site at the moment of scrapping:
Article 1
Article 2
Then we run scrapy again after some time and site will look like this:
Article 1
Article 2
Article 3
Article 4
Can it just append article 3 and 4 to file/db table and stop scraping?
- karma_fountain 9y agoThis project might give you some ideas https://github.com/TeamHG-Memex/scrapy-crawl-once https://github.com/TeamHG-Memex/scrapy-crawl-once. That guy has quite a few scraping projects.