4 ms·
I'm surprised that people are scraping their content via bots, given that it's licensed under CC and freely available to download. (http://blog.stackoverflow.co
by d_r 15y ago
I'm surprised that people are scraping their content via bots, given that it's licensed under CC and freely available to download. (http://blog.stackoverflow.com/category/cc-wiki-dump/ http://blog.stackoverflow.com/category/cc-wiki-dump/)
- Tyrannosaurs 15y agoI'm guessing that these are people who either want something more up to date than the data dump (which is monthly if if I recall) or who have scraping infrastructure set up and working and it's easier to use that than to work with the schema.
- ivoflipse 15y agoIt's bi-monthly nowadays. Even so, you can use the API instead, its there for a reason. Its too bad for this guy that he simply can't access SO the normal way, but that's not SO's fault
- Tyrannosaurs 15y agoI agree. SO is great but we all need to be realistic about situations where for one reason or another we're edge cases.
- drdaeman 15y agoI'm a bit surprised they just block the adresses, nor rate-limiting them.