10 ms·
Scrapy is about as flexible and extensible as you can get... Care to elaborate on "frustratingly inflexible"?
by ddebernardy 11y ago
Scrapy is about as flexible and extensible as you can get... Care to elaborate on "frustratingly inflexible"?
- nasmorn 11y agoImage pipeline converts to jpg automatically. Cannot be disabled without writing all the image code yourself
- staticautomatic 11y agoI do a lot of scraping specific pages and often have to auth, form-fill, refresh, recurse, use a custom SSL/TLS adapter, etc., in order to get what I'm after. I'm sure Scrapy would be great if I just had a giant queue of GET requests. Also, don't get me started on the Reactor.