4 ms·
I really don't understand why anyone would bother writing and using a web scraper when an API exists. Does the API not provide all the same data/functions as t
by magduf 7y ago
I really don't understand why anyone would bother writing and using a web scraper when an API exists. Does the API not provide all the same data/functions as the website? Scrapers are a big PITA compared to just using an API: they're much harder to write to be reliable, and they can break at any time, whenever the site makes even the smallest change. APIs avoid all that mess, and make performance far better too (on both sides), since you're only downloading the data you want, not a ton of Javascript and HTML that you don't.
- barrkel 7y agoAPIs are often not as complete as the web interface, since the customer sees the web interface and normally the customer is what drives the revenue model of the company. If pages are driven via an API, then the API is preferable, but publicly facing websites are often a mix of server-side HTML generation and API enrichment, for caching if nothing else.
- magduf 7y agoIn that case it seems that the webmasters complaining about scraping need to make sure their APIs actually provide access to all the same data, if they want people to use the APIs instead of scraping.