3 ms·
I don't think this can beat the speed of a hand-tune crawler. When I write crawlers, I skip rendering page and javascript execution if it isn't needed, which ma
by dikei 13y ago
I don't think this can beat the speed of a hand-tune crawler. When I write crawlers, I skip rendering page and javascript execution if it isn't needed, which massively speed up the crawling process.
- pranade 13y agoThere's definitely things that a custom-built scraper can do more efficiently than kimono, but our focus right now making scraping accessible across a broad enough range of web sites.