3 ms·
I use it to get rendered web pages I want to scrap.
by jmnicolas 6y ago
I use it to get rendered web pages I want to scrap.
- matt_f 6y ago*scrape
- jamil7 6y agoI was doing the same for an article reader / archiver I built but it was slow. Did you ever figure out how to speed it up? Cancelling image and media requests helped but it increased the likelihood of being flagged as a bot.
- jmnicolas 6y agoSorry I never looked at the perf aspect of it since my needs were modest (less than 50 web pages per day). The first thing that would come to my mind is to parallelize it but then it's going to become pretty complex.