8 ms·

Commenters here make this sound like an open and shut case in favor of Craigslist. IANAL, so could someone explain why Google's index of material covered by co

Commenters here make this sound like an open and shut case in favor of Craigslist. IANAL, so could someone explain why Google's index of material covered by copyright is ok and PadMapper's is not?

mootothemax 14y ago
Most likely because Google aren't using their service to compete with, e.g., Craigslist. I also imagine that if you ask Google (especially with a C&D), they'll be more than happy to remove you from their index.
debacle 14y ago
Because if you ask Google to remove you from their index, they will. Rupert Murdoch has threatened this in the past. Each time he's backed down, because he knows that being removed from Google's index is akin to being removed from the Internet.
- bluesnowmonkey 14y ago
  Craigslist doesn't needs to be removed from the search index. They just need to prevent caching of posts, which is what 3Taps was abusing. Noarchive in robots.txt ought to take care of that.
  - IanDrake 14y ago
    To your knowledge, is it against the law to not follow a site's robots.txt? I'm not aware of one, but I am curious.
    - debacle 14y ago
      I don't know if it's illegal, but Google would follow the site's robots.txt.
    - BenOfTomorrow 14y ago
      Ambiguous. See http://news.ycombinator.com/item?id=4286857 http://news.ycombinator.com/item?id=4286857.
- coliveira 14y ago
  Which shows just how much Google profits from their virtual monopoly. If content companies knew that Google would make billions of dollars selling advertisement based on the content that they generated, it would be very hard for Google to pull the massive Internet scrapping that they managed to do from day one. It is interesting that people understand the illegitimacy of taking data from a single content source (Craigslist), but fail to see how unethical is to take data from everyone and sell advertisements based on that.
  - nostromo 14y ago
    Interestingly, Craigslist is being consistent when it comes to not allowing scrapers, including Google. http://www.craigslist.org/robots.txt http://www.craigslist.org/robots.txt edit: oops! they're not actually blocking google from the apartment listings. thanks smackfu.
    - smackfu 14y ago
      Aren't they only disallowing particular pages there? Like apartments are at *.craigslist.org/apa/number.html which doesn't seem blocked by that robots.txt.
- callmevlad 14y ago
  I can imagine some clever hacks where Padmapper uses Google targeted searches and Google's cache exclusively to get Craigslist data. Since Craigslist would never ask Google to remove them from their index, I wonder if Craigslist would still go after those who get their data as proxied by Google.
  - debacle 14y ago
    > Padmapper uses...Google's cache exclusively to get Craigslist data. That's what Padmapper is doing right now. They're being sued for it.
    - callmevlad 14y ago
      I thought Craigslist is suing 3taps because they scrape their site directly?
epoxyhockey 14y ago
I agree that this isn't a simple open and shut case. But, I doubt that PadMapper will take things the distance before rolling over. It's not a new thing for a start-up to use Craigslist to gain an advantage (i.e. AirBnB), but they all seem to eventually be cut-off.
nchuhoai 14y ago
I was always wondering that. In essence, both cases appropriate CL's data for the users benefit, and both tend to send traffic back. I'm currently in a similar situation, where I have to ask myself how much i can scrape/aggregate/visualize 3rd party data that is useful to an user while stil providing value to the 3rd party. I believe ultimately, it is at the 3rd partys (in this case Craigslist) discretion who is allowed to appropriate their data and who not.
jetti 14y ago
Because you specifically grant access to your content using the robots.txt. If you don't want to have your site indexed, then you change your robots.txt. Or like it was said before, you ask Google to remove you from the index.
declan 14y ago
CL allows Google (as a general-purpose search engine) in its TOU. It doesn't allow Padmapper/3taps. And in this context the TOU are enforceable. I wrote about this a few weeks ago here: https://plus.google.com/112961607570158342254/posts/HcCuHVG7rLM https://plus.google.com/112961607570158342254/posts/HcCuHVG7...
unavoidable 14y ago
[Law student here, but I don't know the full details of the claim CL is making] The case doesn't actually seem open and shut at all. CL seems to have a "moral high ground" in that they provide the original service and the platform for users to post information, but the legal ground is more shaky. It is relatively settled in copyright law (which appears to be the main thrust of CL's argument) that effort alone does not award copyright protection[1] (or indeed, any property rights in general[2]). For example, phone listings are not copyrightable, nor is news. The general question is whether there was creative expression (which can include 'creative' presentation or organization). If I were Padmapper, I would argue that classified listings are just a collection of facts, in which case they are not copyrightable. Padmapper does not actually use the listings directly, but extracts the information and puts it in a different format (their interface). On this point I think their case is relatively strong. Where CL might be able to make a better case is their TOU. The TOU forms a contract between CL and their users. In almost all such prominent cases, this type of contract has been found to be enforceable[3]. If this line of reasoning is followed, then Padmapper will probably lose on this point because they agreed not to "copy, aggregate, distribute", etc.[4] If this is a contractual claim the damages would be what CL could have expected if Padmapper had not 'breached' the contract - in this case I'm not sure that amount is particularly large, because Padmapper redirects to CL's site for the actual listings. TLDR: copyright claim seems weak, but the contractual claim might succeed. The trademark stuff seems like a red herring and is CL throwing stuff at the wall to see what sticks. [EDIT: see below, I overlooked the fact that Padmapper is no longer using CL data directly, so they might not be bound by the TOU at all. Same goes for 3Taps since they use Google's cache. Looking stronger for Padmapper] [1] Feist v. Rural, http://en.wikipedia.org/wiki/Feist_v._Rural http://en.wikipedia.org/wiki/Feist_v._Rural [2] INS v. AP, http://en.wikipedia.org/wiki/International_News_Service_v._Associated_Press http://en.wikipedia.org/wiki/International_News_Service_v._A... [3] http://en.wikipedia.org/wiki/Clickwrap http://en.wikipedia.org/wiki/Clickwrap [4] CL Terms of Use (see section 3), http://www.craigslist.org/about/terms.of.use http://www.craigslist.org/about/terms.of.use
- nostromo 14y ago
  Thanks, very interesting!
- 14y ago
nhebb 14y ago
I don't know if it's open and shut either - but not because of the search analogy. That's covered in the CL ToS, sections 3 and 5. (http://www.craigslist.org/about/terms.of.use http://www.craigslist.org/about/terms.of.use) My problem is with section 3. CONTENT AND CONDUCT, part a, where they first state "CL does not control, is not responsible for and makes no representations or warranties with respect to any user content." Then it goes on to state that posters assign a bevy of licensing rights to CL for the use of the content. How can they claim copyright to content and at the same time aver that they don't control it?
- unavoidable 14y ago
  This is where the lawsuit gets _legally_ interesting. Craigslist can't really claim copyright over the listings themselves (see post above), but they can certainly restrict user rights through their terms of use (which is a contract). Generally, contracts can be used to reduce a set of rights a person has against another (e.g. waivers), but it cannot grant rights that they did not have in the first place. So the first part of that statement is akin to waiver, so that someone can't sue CL for what the listings say. The second part is a claim on copyright and licensing, which they may or may not have.

Commenters here make this sound like an open and shut case in favor of Craigslist. IANAL, so could someone explain why Google's index of material covered by copyright is ok and PadMapper's is not?

mootothemax 14y ago

Most likely because Google aren't using their service to compete with, e.g., Craigslist. I also imagine that if you ask Google (especially with a C&D), they'll be more than happy to remove you from their index.

debacle 14y ago

Because if you ask Google to remove you from their index, they will. Rupert Murdoch has threatened this in the past. Each time he's backed down, because he knows that being removed from Google's index is akin to being removed from the Internet.

bluesnowmonkey 14y ago
Craigslist doesn't needs to be removed from the search index. They just need to prevent caching of posts, which is what 3Taps was abusing. Noarchive in robots.txt ought to take care of that.
- IanDrake 14y ago
  To your knowledge, is it against the law to not follow a site's robots.txt? I'm not aware of one, but I am curious.
  - debacle 14y ago
    I don't know if it's illegal, but Google would follow the site's robots.txt.
  - BenOfTomorrow 14y ago
    Ambiguous. See http://news.ycombinator.com/item?id=4286857 http://news.ycombinator.com/item?id=4286857.
coliveira 14y ago
Which shows just how much Google profits from their virtual monopoly. If content companies knew that Google would make billions of dollars selling advertisement based on the content that they generated, it would be very hard for Google to pull the massive Internet scrapping that they managed to do from day one. It is interesting that people understand the illegitimacy of taking data from a single content source (Craigslist), but fail to see how unethical is to take data from everyone and sell advertisements based on that.
- nostromo 14y ago
  Interestingly, Craigslist is being consistent when it comes to not allowing scrapers, including Google. http://www.craigslist.org/robots.txt http://www.craigslist.org/robots.txt edit: oops! they're not actually blocking google from the apartment listings. thanks smackfu.
  - smackfu 14y ago
    Aren't they only disallowing particular pages there? Like apartments are at *.craigslist.org/apa/number.html which doesn't seem blocked by that robots.txt.
callmevlad 14y ago
I can imagine some clever hacks where Padmapper uses Google targeted searches and Google's cache exclusively to get Craigslist data. Since Craigslist would never ask Google to remove them from their index, I wonder if Craigslist would still go after those who get their data as proxied by Google.
- debacle 14y ago
  > Padmapper uses...Google's cache exclusively to get Craigslist data. That's what Padmapper is doing right now. They're being sued for it.
  - callmevlad 14y ago
    I thought Craigslist is suing 3taps because they scrape their site directly?

epoxyhockey 14y ago

I agree that this isn't a simple open and shut case. But, I doubt that PadMapper will take things the distance before rolling over. It's not a new thing for a start-up to use Craigslist to gain an advantage (i.e. AirBnB), but they all seem to eventually be cut-off.

nchuhoai 14y ago

I was always wondering that. In essence, both cases appropriate CL's data for the users benefit, and both tend to send traffic back. I'm currently in a similar situation, where I have to ask myself how much i can scrape/aggregate/visualize 3rd party data that is useful to an user while stil providing value to the 3rd party. I believe ultimately, it is at the 3rd partys (in this case Craigslist) discretion who is allowed to appropriate their data and who not.

jetti 14y ago

Because you specifically grant access to your content using the robots.txt. If you don't want to have your site indexed, then you change your robots.txt. Or like it was said before, you ask Google to remove you from the index.

declan 14y ago

CL allows Google (as a general-purpose search engine) in its TOU. It doesn't allow Padmapper/3taps. And in this context the TOU are enforceable. I wrote about this a few weeks ago here: https://plus.google.com/112961607570158342254/posts/HcCuHVG7rLM https://plus.google.com/112961607570158342254/posts/HcCuHVG7...

unavoidable 14y ago

[Law student here, but I don't know the full details of the claim CL is making] The case doesn't actually seem open and shut at all. CL seems to have a "moral high ground" in that they provide the original service and the platform for users to post information, but the legal ground is more shaky. It is relatively settled in copyright law (which appears to be the main thrust of CL's argument) that effort alone does not award copyright protection[1] (or indeed, any property rights in general[2]). For example, phone listings are not copyrightable, nor is news. The general question is whether there was creative expression (which can include 'creative' presentation or organization). If I were Padmapper, I would argue that classified listings are just a collection of facts, in which case they are not copyrightable. Padmapper does not actually use the listings directly, but extracts the information and puts it in a different format (their interface). On this point I think their case is relatively strong. Where CL might be able to make a better case is their TOU. The TOU forms a contract between CL and their users. In almost all such prominent cases, this type of contract has been found to be enforceable[3]. If this line of reasoning is followed, then Padmapper will probably lose on this point because they agreed not to "copy, aggregate, distribute", etc.[4] If this is a contractual claim the damages would be what CL could have expected if Padmapper had not 'breached' the contract - in this case I'm not sure that amount is particularly large, because Padmapper redirects to CL's site for the actual listings. TLDR: copyright claim seems weak, but the contractual claim might succeed. The trademark stuff seems like a red herring and is CL throwing stuff at the wall to see what sticks. [EDIT: see below, I overlooked the fact that Padmapper is no longer using CL data directly, so they might not be bound by the TOU at all. Same goes for 3Taps since they use Google's cache. Looking stronger for Padmapper] [1] Feist v. Rural, http://en.wikipedia.org/wiki/Feist_v._Rural http://en.wikipedia.org/wiki/Feist_v._Rural [2] INS v. AP, http://en.wikipedia.org/wiki/International_News_Service_v._Associated_Press http://en.wikipedia.org/wiki/International_News_Service_v._A... [3] http://en.wikipedia.org/wiki/Clickwrap http://en.wikipedia.org/wiki/Clickwrap [4] CL Terms of Use (see section 3), http://www.craigslist.org/about/terms.of.use http://www.craigslist.org/about/terms.of.use

nostromo 14y ago
Thanks, very interesting!
14y ago

nhebb 14y ago

I don't know if it's open and shut either - but not because of the search analogy. That's covered in the CL ToS, sections 3 and 5. (http://www.craigslist.org/about/terms.of.use http://www.craigslist.org/about/terms.of.use) My problem is with section 3. CONTENT AND CONDUCT, part a, where they first state "CL does not control, is not responsible for and makes no representations or warranties with respect to any user content." Then it goes on to state that posters assign a bevy of licensing rights to CL for the use of the content. How can they claim copyright to content and at the same time aver that they don't control it?

unavoidable 14y ago
This is where the lawsuit gets _legally_ interesting. Craigslist can't really claim copyright over the listings themselves (see post above), but they can certainly restrict user rights through their terms of use (which is a contract). Generally, contracts can be used to reduce a set of rights a person has against another (e.g. waivers), but it cannot grant rights that they did not have in the first place. So the first part of that statement is akin to waiver, so that someone can't sue CL for what the listings say. The second part is a claim on copyright and licensing, which they may or may not have.