3 ms·
I am sure the orignal author does as well. This is more a question of "fair use", not robots.txt
by jcampbell1 14y ago
I am sure the orignal author does as well. This is more a question of "fair use", not robots.txt
- chii 14y agoIndeed - and i believe scraping data off a publically accessible webpage (e.g., one where you do not require registration and login/password) falls under fair use, provided you do not take up more bandwidth/resources than the average user of that site. I recall there being some precedent for this sort of fair use - something like a phone directory - the information is not copyrighted, but the arrangement and layout is. So hence, you can't just iframe a site and present it, but obtaining the data, and deriving a new work from it should fall under fair use.
- benmccann 14y agoThat court case was Feist v. Rural. The court ruled that copying the listings in a phone book was permitted because facts cannot be copyrighted.