4 ms·
There's one significant difference between an API and site scraping: versioning. A documented API that doesn't come with some form of commitment not to break i
by barrkel 13y ago
There's one significant difference between an API and site scraping: versioning.
A documented API that doesn't come with some form of commitment not to break it is little better than web scraping.
Web scraping, meanwhile, is subject to breakage at every whim of the web site designers.
- timje1 13y agoThis article doesn't recommend web scraping - he discusses making URLs for your 'API' resources and your human-consumable resources shared, with different 'accept:' content types.
- randomdata 13y agoWhich is the parent's point. The actual content type you are sending is irrelevant. HTML, JSON, CSV, it is all the same from a computer's point of view. What is relevant is that the content's structure remains stable. If you are sharing URLs, the structure of the 'API formats' and the structure of the HTML page should be consistent with each other, which means that any big changes to the HTML form could potentially break the API across every format in order to maintain that consistency. If you are going to make your JSON structure stable but then radically change the structure of your HTML page at the same address, then the URI becomes a misnomer and you've lost the reason for having the same resource address for multiple content types in the first place.
- zimbatm 13y agoNot really. In both cases it's not desirable for the URLs to change in case a client/other website is linking to it. Versioning is just adding more URLs in the same namespace.
- martindale 13y agoThere's a great essay by Timothy Berners-Lee on changing URLs: http://www.w3.org/Provider/Style/URI.html http://www.w3.org/Provider/Style/URI.html
- barrkel 13y agoURLs are not the only component of web APIs.
- MattBearman 13y agoUnless I'm misunderstanding you, URLs do not have to change to break a scraper. If the HTML of the site is significantly changed (ie: redesigned) it will likely break any scrapers using it, without having to change any URLs.
- stefan_kendall 13y agoAPIs change out from underneath you. Code can't last forever, even with versioning.
- barrkel 13y agoOf course the guarantee isn't absolute; that would be silly. But so is thinking that this isn't an absolutely crucial distinguishing factor. I can run programs written 17 years ago on my current Windows box, not even needing a VM. In computer terms, that's almost forever, and it was made possible by Microsoft's commitment to API compatibility. API versioning is a mindset. It means that once you've published something, you can't change it without carefully managing the effects. It potentially means maintaining bug compatibility; it sometimes means detecting the client of the API, and specifically accommodating client bugs. Other APIs, even internal APIs used by rich web apps to communicate with their back ends, are not written with the same mindset. And rightly so; the constraints that come along with a public API are not to be taken lightly, and do hamper agility.
- prodigal_erik 13y agoCode can last forever; there are people playing thirty-year-old games via emulation. The only problem is that we now have a plague of people who don't take their responsibilities seriously as contributors to the Web, and they randomly break their own resources out of laziness or ignorance.
- AznHisoka 13y agoTell that to Facebook and Twitter. Both are notorious for making changes to their API without informing their developers , especially the former. With site scraping, at least they have a better chance of telling you when a redesign is coming. And b/c they rely so much on CDN's, JS, caching, etc, changes to their site structure probably don't change as much as their API.