4 ms·
Depends on a country. And in practice on how much money the people behind the website can throw at the lawyers when annoyed. For the US you may want to read ab
by viraptor 8y ago
Depends on a country. And in practice on how much money the people behind the website can throw at the lawyers when annoyed.
For the US you may want to read about https://en.wikipedia.org/wiki/Aaron_Swartz https://en.wikipedia.org/wiki/Aaron_Swartz
- Petrakis 8y agoAlso depends on the ToS of each page, and their robots.txt should allow or disallow crawling