3 ms·
That's what I meant by narrow...a known set of sites and data you want to extract. I imagine, for example, building on the SERP example might hit a wall if you
by tyingq 6y ago
That's what I meant by narrow...a known set of sites and data you want to extract.
I imagine, for example, building on the SERP example might hit a wall if you added logged in vs not logged SERPS, iterating over carousel data, reading advertisement data etc.
- daolf 6y agolog in wall can easily be bypassed with an HTTP client by setting the correct auth header. From what I can observe, 2/3 websites can be scraped without using a headless browser.