24 ms·
This issue is just an instance of a big problem: there is not such thing as a data democracy, only a theory. It's not about Scraperwiki only, the SLA for the fr
by wslh 12y ago
This issue is just an instance of a big problem: there is not such thing as a data democracy, only a theory. It's not about Scraperwiki only, the SLA for the free Twitter API is very restrictive (i.e: number of calls per hour).
Twitter (and others) works like a cartel, imagine that you want to pay to increase the number of API calls per hour, you don't have a straightforward way to do it. Twitter has a lot to learn from what Google did almost a decade ago: make it easy to pay and work with your service instantly (AdWords).
- theworst 12y agoA little off topic, but I'm curious where the HN crowd lies on the ethics of scraping. I'm a bit on the copyleft side -- I say, if something is made public, it's fair game. If you want to protect your data from being scraped, you very easily indicate that by putting it behind a login. But in my opinion, anything google is allowed to access and put in the index, I can access.
- wslh 12y agoI think at the startup level you can "hack the system". The issue comes when you grow.
- theworst 12y agoAs a practical, or ethical issue? IE, is it just as "wrong" in your opinion to scrape sites, regardless of size or profitability? Or are you espousing the practical argument that it's fine because you can get away with it when flying under the radar?