4 ms·
It is very slow. But personally, I prefer to write my crawlers in Go (custom code, not Colly).
by estebarb 5y ago
It is very slow. But personally, I prefer to write my crawlers in Go (custom code, not Colly).
- zatarc 5y agoTry Parsel: https://github.com/scrapy/parsel https://github.com/scrapy/parsel It's way faster and has better support for CSS selectors.
- moehm 5y agoWhat's your problem with Colly? [0] [0] http://go-colly.org/ http://go-colly.org/
- estebarb 5y agoMostly that I started my crawler before learning about Colly and it didn't make sense to rewrite the code. By "not Colly" I just wanted to remark that in Go is relatively easy to write a crawler from scratch.
- thegeekpirate 5y ago> But personally, I prefer to write my crawlers in Go (custom code, not Colly). This is my current setup as well, been scraping on and off for 20+ years now.