3 ms·
One problem that the author notes is that so much rendering is done client side via javascript. The flip side to this is that very often you find that the data
by andrewla 2y ago
One problem that the author notes is that so much rendering is done client side via javascript.
The flip side to this is that very often you find that the data populating the site is in a very simple JSON format to facilitate easy rendering, ironically making the scraping process a lot more reliable.
- sakisv 2y agoInitially that's what I wanted to do, but the first supermarket I did is sending back HTML rendered on the server side, so I abandonded this approach for the sake of "consistency". Lately I've been thinking to bite the bullet and Just Do It, but since it's working I'm a bit reluctant to touch it.
- andrewla 2y agoFor your purposes scraping the user-visible site probably makes the most sense since in the end, their users' eyes are the target. I am typically doing one-off scraping and for that, an undocumented but clean JSON api makes things so much easier, so I've grown to enjoy sites that are unnecessarily complex in their rendering.