3 ms·
Ask HN: Is API scraping more common now than page scraping?
Given that many/most sites are SPA-style now that make structured http requests to retrieve their data (after initial page load), does modern scraping target underlying api's now, or is it still centered around parsing markup? If the latter, why?
- gcatalfamo 7y agoThere isn’t a general rule. It depends on the website architecture. If there is an API you can use it is better for you and less tolling on the website (given you know what you are doing), otherwise scraping it is.
- kakapumar 7y agodoesn't have any sense to make manual crawling and parsing, having tools like crawlera or proxycrawl makes it super easy to get data in API style way. So why complicate?