3 ms·
Try to imagine how to solve the second example of the sukhoi README.md using scrapy, you'll notice you'll end up with some kind of obscure logic to achieve that
by iogf 9y ago
Try to imagine how to solve the second example of the sukhoi README.md using scrapy, you'll notice you'll end up with some kind of obscure logic to achieve that json structure thats outputed by the second example in sukhoi's README.md.
- bbernoulli 9y agoFWIW I don't believe this would be overly convoluted in scrapy. I'd probably scrape the tags and quotes in one pass... Also, generator expressions would make the examples more readable IMO. self.extend((tag, QuoteMiner(self.geturl(href))) for tag, href in self.acc)
- iogf 9y agoI would like to see that in scrapy. I think you may have a point about the generators, yea.