2 ms·
IANAL but as far as I understand the current legal status (in the US) a change in robots.txt or terms and conditions is not binding for web scrapers since the d
by StrauXX 2y ago
IANAL but as far as I understand the current legal status (in the US) a change in robots.txt or terms and conditions is not binding for web scrapers since the data is publicly accessible. Neither does displaying a banner "By using this site you accept our terms and conditions" change anything about that. The only thing that can make these kinds of terms binding is if the data is only accessible after proactively accepting terms. For instance by restricting the website until one has created an account. Linkedin lost a case against a startup scraping and indexing their data because of that a few years ago.
- jpalomaki 2y agoQuite sure they are also enforcing these with some technical measures to limit scraping.
- renlo 2y agoAs was LinkedIn, who was forced to rate stop limiting / IP-banning scrapers for public pages.
- therealdrag0 2y agoReally? That seems strange.
- altdataseller 2y agoMany other websites enforce rate limiting or IP banning for public pages (using products like Cloudflare). Why is this not legal?
- qingcharles 2y agoAt the federal level; but states have their own laws. For instance, it can get you 5 years in prison in Illinois to violate a web site ToS. https://www.ilga.gov/legislation/ilcs/ilcs4.asp?DocName=072000050HArt%2E+17%2C+Subdiv%2E+30&ActID=1876&ChapterID=53&SeqStart=59300000&SeqEnd=60000000 https://www.ilga.gov/legislation/ilcs/ilcs4.asp?DocName=0720...
- redcobra762 2y agoHas anyone ever successfully been prosecuted for violating this statute?
- qingcharles 2y agoI don't know. That data is really hard to put together.
- quink 2y agoThat’s a blink and you miss it joke of the highest calibre - well done!