4 ms·

1) I assume b/c libraries purchase the book it's not equivalent. Do they purchase a subscription to the NYT and then lend it out to people? From what I gather o

by geokon 5y ago

1) I assume b/c libraries purchase the book it's not equivalent. Do they purchase a subscription to the NYT and then lend it out to people? From what I gather on their website it doesn't seem like it 2) I'm not sure why you just made that up .. https://archive.is/faq https://archive.is/faq "Why does archive.is not obey robots.txt? Because it is not a free-walking crawler, it saves only one page acting as a direct agent of the human user. Such services don't obey robots.txt (e.g. Google Feedfetcher, screenshot- or pdf-making services, isup.me, …)"

kijin 5y ago
1) There doesn't seem to be much difference in terms of "lost sales" (a common metric in copyright disputes) when a library purchases 1 copy of a book and lends it out to 100 people, versus when the Internet Archive purchases 0 copies and lets 100 people view it. Edit: Besides, the way the FAQ describes it ("acting as a direct agent of the human user"), we can't preclude the possibility that they're getting content through a paying subscriber's account. The Internet Archive used to collaborate with Alexa, which is known to have used browser plugins to collect data from users' browsers. 2) It used to work reliably a few years ago, but it seems that they changed their policies and my knowledge is outdated. Sorry about that. In any case, the New York Times is perfectly capable of putting up a paywall on any subset of visitors they want, but chose to turn a blind eye on a broad set of crawlers -- presumably because they want the Google traffic. That could be seen as tacit consent to being crawled, so I'm not sure how much legal ground they're going to have if they later decide to sue the crawlers.
hoppyhoppy2 5y ago
I think part of the confusion is that they were referring to archive.org, which is a service of the Internet Archive. Archive.is is a separate site run by someone else, operating under different guidelines. There is no need to be rude over a misunderstanding.
- geokon 5y ago
  well I didn't mean to be rude. but it was an off the cuff remark that was wrong. People say things without posting a source and that's how disinformation starts... first thing that came up in search suggests archive.org also is moving away from listening to robot.text files (granted this is an old blog post) https://blog.archive.org/2017/04/17/robots-txt-meant-for-search-engines-dont-work-well-for-web-archives/ https://blog.archive.org/2017/04/17/robots-txt-meant-for-sea... I couldn't find anything stating their policy clearly. but maybe I didn't search hard enough. I find it hard to believe news papers wouldn't block them if it was that easy

by geokon 5y ago

kijin 5y ago

1) There doesn't seem to be much difference in terms of "lost sales" (a common metric in copyright disputes) when a library purchases 1 copy of a book and lends it out to 100 people, versus when the Internet Archive purchases 0 copies and lets 100 people view it. Edit: Besides, the way the FAQ describes it ("acting as a direct agent of the human user"), we can't preclude the possibility that they're getting content through a paying subscriber's account. The Internet Archive used to collaborate with Alexa, which is known to have used browser plugins to collect data from users' browsers. 2) It used to work reliably a few years ago, but it seems that they changed their policies and my knowledge is outdated. Sorry about that. In any case, the New York Times is perfectly capable of putting up a paywall on any subset of visitors they want, but chose to turn a blind eye on a broad set of crawlers -- presumably because they want the Google traffic. That could be seen as tacit consent to being crawled, so I'm not sure how much legal ground they're going to have if they later decide to sue the crawlers.

hoppyhoppy2 5y ago

I think part of the confusion is that they were referring to archive.org, which is a service of the Internet Archive. Archive.is is a separate site run by someone else, operating under different guidelines. There is no need to be rude over a misunderstanding.

geokon 5y ago
well I didn't mean to be rude. but it was an off the cuff remark that was wrong. People say things without posting a source and that's how disinformation starts... first thing that came up in search suggests archive.org also is moving away from listening to robot.text files (granted this is an old blog post) https://blog.archive.org/2017/04/17/robots-txt-meant-for-search-engines-dont-work-well-for-web-archives/ https://blog.archive.org/2017/04/17/robots-txt-meant-for-sea... I couldn't find anything stating their policy clearly. but maybe I didn't search hard enough. I find it hard to believe news papers wouldn't block them if it was that easy