3 ms·
If you are just doing one or two pages, say you want to get weather for your location, then requests is sufficient. But if you want to do many pages where you m
by aynyc 5y ago
If you are just doing one or two pages, say you want to get weather for your location, then requests is sufficient. But if you want to do many pages where you might want to scan and follow, requests gets tedious very quickly.
- r_singh 5y agoIf you’re a web developer not really, rather than worrying about storage backendes, spiders, yielding and managing loops and items, you could just host a DRF or Flask API with your scrapers (written in Requests+lxml) initiated with an API request. I guess it’s a matter of preference