4 ms·

There are a great many sites that degrade gracefully when JS support is not available. It makes absolutely no sense to waste the resources required to run a ful

by sp0rk 9y ago

There are a great many sites that degrade gracefully when JS support is not available. It makes absolutely no sense to waste the resources required to run a full headless browser when simple HTTP requests will retrieve the same information faster, more efficiently, and in a way that's easier to parallelize.

xur17 9y ago
A lot of times you can also watch the api calls JS pages (or apps) make and retrieve nice structured json data. I personally avoid executing js unless it's necessary, as it adds more complexity, and is noticeably more brittle.
- jordanpg 9y ago
  Using an undocumented API, however, carries significant risk for production operations.
  - flukus 9y ago
    If you're web scraping then you've already decided that this risk is worthwhile, it's already an undocumented API.
jordanpg 9y ago
Yes, but a great many sites don't, and for those, you need Selenium + browser, full stop.