2 ms·
Scrapy is written in Python. This looks like a Ruby focused article. It's even written in the title, no need to actually go and read it. I'd say your suggestion
by klibertp 12y ago
Scrapy is written in Python. This looks like a Ruby focused article. It's even written in the title, no need to actually go and read it. I'd say your suggestion is simply off-topic here.
As for Scrapy itself, it's a big framework, written on top of even bigger framework which is probably better described as a platform at this point. I've used Scrapy in a couple of projects and I also worked with Twisted before, which made things significantly easier for me, and it still was quite a bit of a hassle to set things up. IIRC configuring a pipeline for saving images to disk with their original names was kind of a nightmare. It does perform extremely well and scales to insane workloads, but I would never use it for simple scrapper for a single site. For those requests+lxml work extremely well.