1) There doesn't seem to be much difference in terms of "lost sales" (a common metric in copyright disputes) when a library purchases 1 copy of a book and lends

1) There doesn't seem to be much difference in terms of "lost sales" (a common metric in copyright disputes) when a library purchases 1 copy of a book and lends it out to 100 people, versus when the Internet Archive purchases 0 copies and lets 100 people view it. Edit: Besides, the way the FAQ describes it ("acting as a direct agent of the human user"), we can't preclude the possibility that they're getting content through a paying subscriber's account. The Internet Archive used to collaborate with Alexa, which is known to have used browser plugins to collect data from users' browsers. 2) It used to work reliably a few years ago, but it seems that they changed their policies and my knowledge is outdated. Sorry about that. In any case, the New York Times is perfectly capable of putting up a paywall on any subset of visitors they want, but chose to turn a blind eye on a broad set of crawlers -- presumably because they want the Google traffic. That could be seen as tacit consent to being crawled, so I'm not sure how much legal ground they're going to have if they later decide to sue the crawlers.