3 ms·
Everything that can't be handled with curl or beautifulsoup is probably not worth the effort.
by gsich 9y ago
Everything that can't be handled with curl or beautifulsoup is probably not worth the effort.
- _eht 9y agoAnything that can't be loaded over 2400 baud is probably not worth the effort.
- gsich 9y agoSomewhat correct, although an extreme example. One should always try to design websites with low bandwidth users in mind.
- emodendroket 9y agoLots of pages have the content you'd want to scrape injected with JS; headless Chrome would seem to solve that problem.
- imtringued 9y agoDon't most SPAs have some kind of internal API that is easier to work with than the HTML?