4 ms·

Does this effectively mean Padmapper can use CL data once again?

pyre 14y ago
The exclusive license effectively meant that you couldn't cross-post to both CraigsList and PadMapper because you had granted exclusivity to CraigsList.
- BadassFractal 14y ago
  Ok makes sense, so that legitimizes something like PadLister. Does this however enable one to scrape CL for data?
  - danso 14y ago
    ...this does not legitimize anything in particular. It says it's OK for users to post ads on multiple sites, not that other sites can scrape CL's user content. This is akin to freelance contract work: if you're a news photographer, there's usually a clause saying that while you can own your photos, you may not sell them to your client's competitors (at least right away). So CL is saying to its users: Go ahead and copy and paste your stuff to other sites. So, not a huge difference by any means in what most users assumed.
    - BadassFractal 14y ago
      PadMapper was doing the cross-posting to PadList to slowly build its user base. I however did not realize that Craigslist frowned upon that until the change in the agreement.
      - iamwil 14y ago
        PadMapper doesn't take CL listings and put it into Padlister. What's listed on Padlister was listed by users. PadMapper puts CL listings and Padlister listings on the same map.
waterlesscloud 14y ago
CL has also removed themselves from search engines, so PadMapper can no longer use the google cache workaround. I don't think they can go back to scraping either, but I'm not certain of that aspect of it. This may be the real reason behind the change. Perhaps CL just doesn't need it any more.
- danso 14y ago
  I think this is correct, though the EFF post is too vague, and doesn't quote the passage in its current state. Also, CL has not removed itself from search engines. They're using NOARCHIVE.
  - dredmorbius 14y ago
    Last I checked, robots.txt excludes all crawlers from at least several major classifications, including housing listings (/hhh): User-agent: * Disallow: /cgi-bin Disallow: /cgi-secure Disallow: /forums Disallow: /search Disallow: /res/ Disallow: /post Disallow: /email.friend Disallow: /eaf Disallow: /reply Disallow: /?flagCode Disallow: /ccc Disallow: /hhh Disallow: /sss Disallow: /bbb Disallow: /ggg Disallow: /jjj Disallow: /*rss$ http://www.craigslist.org/robots.txt http://www.craigslist.org/robots.txt 'jjj' is jobs, 'ggg' is gigs, 'bbb' is services, 'sss' is for sale, 'ccc' is community, 'res' is resumes.
    - danso 14y ago
      Right...but not apartment listings (aap) http://newyork.craigslist.org/aap/ http://newyork.craigslist.org/aap/ http://newyork.craigslist.org/roo/ http://newyork.craigslist.org/roo/ http://newyork.craigslist.org/sub/ http://newyork.craigslist.org/sub/ http://www.google.com/search?client=safari&rls=en&q=bronx+all+apartments+classifieds&ie=UTF-8&oe=UTF-8 http://www.google.com/search?client=safari&rls=en&q=...
      - dredmorbius 14y ago
        Point. I mistakenly assumed /aaa covered all of housing. Looks like the same principle applies to a number of other subcategories. So while overview indices aren't spiderable, subindices are.
radtad 14y ago
My understanding is that this is a separate legal issue. CL's complaint against PadMapper is for scraping, not with users reposting on PadLister.

pyre 14y ago

The exclusive license effectively meant that you couldn't cross-post to both CraigsList and PadMapper because you had granted exclusivity to CraigsList.

BadassFractal 14y ago
Ok makes sense, so that legitimizes something like PadLister. Does this however enable one to scrape CL for data?
- danso 14y ago
  ...this does not legitimize anything in particular. It says it's OK for users to post ads on multiple sites, not that other sites can scrape CL's user content. This is akin to freelance contract work: if you're a news photographer, there's usually a clause saying that while you can own your photos, you may not sell them to your client's competitors (at least right away). So CL is saying to its users: Go ahead and copy and paste your stuff to other sites. So, not a huge difference by any means in what most users assumed.
  - BadassFractal 14y ago
    PadMapper was doing the cross-posting to PadList to slowly build its user base. I however did not realize that Craigslist frowned upon that until the change in the agreement.
    - iamwil 14y ago
      PadMapper doesn't take CL listings and put it into Padlister. What's listed on Padlister was listed by users. PadMapper puts CL listings and Padlister listings on the same map.

waterlesscloud 14y ago

CL has also removed themselves from search engines, so PadMapper can no longer use the google cache workaround. I don't think they can go back to scraping either, but I'm not certain of that aspect of it. This may be the real reason behind the change. Perhaps CL just doesn't need it any more.

danso 14y ago
I think this is correct, though the EFF post is too vague, and doesn't quote the passage in its current state. Also, CL has not removed itself from search engines. They're using NOARCHIVE.
- dredmorbius 14y ago
  Last I checked, robots.txt excludes all crawlers from at least several major classifications, including housing listings (/hhh): User-agent: * Disallow: /cgi-bin Disallow: /cgi-secure Disallow: /forums Disallow: /search Disallow: /res/ Disallow: /post Disallow: /email.friend Disallow: /eaf Disallow: /reply Disallow: /?flagCode Disallow: /ccc Disallow: /hhh Disallow: /sss Disallow: /bbb Disallow: /ggg Disallow: /jjj Disallow: /*rss$ http://www.craigslist.org/robots.txt http://www.craigslist.org/robots.txt 'jjj' is jobs, 'ggg' is gigs, 'bbb' is services, 'sss' is for sale, 'ccc' is community, 'res' is resumes.
  - danso 14y ago
    Right...but not apartment listings (aap) http://newyork.craigslist.org/aap/ http://newyork.craigslist.org/aap/ http://newyork.craigslist.org/roo/ http://newyork.craigslist.org/roo/ http://newyork.craigslist.org/sub/ http://newyork.craigslist.org/sub/ http://www.google.com/search?client=safari&rls=en&q=bronx+all+apartments+classifieds&ie=UTF-8&oe=UTF-8 http://www.google.com/search?client=safari&rls=en&q=...
    - dredmorbius 14y ago
      Point. I mistakenly assumed /aaa covered all of housing. Looks like the same principle applies to a number of other subcategories. So while overview indices aren't spiderable, subindices are.

radtad 14y ago

My understanding is that this is a separate legal issue. CL's complaint against PadMapper is for scraping, not with users reposting on PadLister.