3 ms·
People who are saying that the author should have used an API are missing the point; sometimes the only way to gather data is by resorting to scraping, and this
by depressedpanda 7y ago
People who are saying that the author should have used an API are missing the point; sometimes the only way to gather data is by resorting to scraping, and this article details how one could go about doing that, by using an interesting target: HN.
To the author: Good job, I'm sure many people will find your article useful! I wish I would've had this article at hand a couple of years ago when I needed to aggregate data from several websites that provided no API.
- minimaxir 7y ago> sometimes the only way to gather data is by resorting to scraping Nowadays, web scraping has a nonzero risk of hitting legal issues for sites when "the only way to gather data is by resulting to scraping", especially when an API already exists with a ToS/Guidelines on how the data should be obtained and used. And even moreso if the data itself has monetary value.
- philipkiely 7y agoThank you!!