4 ms·
A good project, how does it compare to python based scrapy.org. For admin you can use scrapyhub open source or also another project https://github.com/Gerapy/G
by dragonsh 7y ago
A good project, how does it compare to python based scrapy.org.
For admin you can use scrapyhub open source or also another project https://github.com/Gerapy/Gerapy https://github.com/Gerapy/Gerapy
- chrischen 7y agoScrapy is a web crawler. This project is a web crawler management UI/platform, so it presumably manages your scrapy crawlers/instances and schedules them.
- dragonsh 7y agoSo if I understand it correct Crawlab is another simple easy to use admin for managing web crawlers, one still needs to use scrapy or write their own crawlers. It should be similar to the admin tool I mentioned in my earlier comment and at: https://github.com/topics/scrapy-ui https://github.com/topics/scrapy-ui https://github.com/topics/scrapyd-ui https://github.com/topics/scrapyd-ui
- tikazyq 7y agoThere are a couple of crawler management projects: scrapydweb, spiderkeeper, gerapy, crawlab. The first three are based on scrapyd.