4 ms·

Craigslist doesn't have an open system. Craigslist blocks ip's of those accessing there system through a web application and they send out lots of cease and des

by gscott 19y ago

Craigslist doesn't have an open system. Craigslist blocks ip's of those accessing there system through a web application and they send out lots of cease and desist notices. It just is not worth the time to develop something that might even benefit Craigslist. Piggybacking on bigger sites is a good idea, just find one that is interested. Paypal did for eBay and it worked out for them, There is a lot of room to develop things using ebay. For example, maybe a craigslist format but for ebay auctions.

optimal 19y ago
"Craigslist doesn't have an open system. Craigslist blocks ip's of those accessing there system through a web application and they send out lots of cease and desist notices" Screw it. Right now, a group here should build a newer, better Craigslist. Enough of you guys are in touch with the SV/Bay scene to bring the eyeballs required to kick it off, because as I see it, it's not a technical problem but rather a marketing one: what you need is a brief but intense "attention burst" to launch something like this. The bleeding-edge users are all here, or only one or two degrees removed, and could make it work. [Don't look at me, I'm stuck on the East Coast.]
mlinsey 19y ago
"Paypal did for eBay and it worked out for them", which is true but it should also be noted that it worked only after eBay launched a competing product that PayPal was able to beat despite the enormous distribution advantages that an eBay-run competitor had.
aaronsw 19y ago
That's a pretty serious claim; any pointers? If you've got some I'll be sure to nag Craig about it next time I see him.
- fragmede 19y ago
  listpic? http://mashable.com/2007/06/08/listpic-craigslist/ http://mashable.com/2007/06/08/listpic-craigslist/
  - vesterr 19y ago
    Ha, with EC2 you can get around that now. Separate your website IP from your crawling IP, and every time CL blocks an IP, switch to a new one. Eventually they'll have to block the entire AWS range, but that's okay, you can crawl over cablemodem/DSL connections that use DHCP. What are they going to do, block Verizon, Comcast and Time Warner? Then you can get around referer [sic] blocks for your links to CL that the user clicks on by using a redirect, I think. You can slurp the entirety of CL daily without causing them traffic problems. I mean it's equivalent to each page getting one page view per day, which is nothing. Just keep track of URLs so you only slurp new content, and serve thumbnails off your own hosts (it's fair use).
    - nuggien 19y ago
      Why would you go to all that effort if they don't want to play with you in the first place? If they really wanted to block you, they'll do it through their lawyers.
      - vesterr 19y ago
        That stuff is really easy. And you may convince them to join you after all, like YouTube and the media companies they parasite off of.
        nuggien 19y ago
        I didn't say it was hard. It's also very easy to differentiate between a crawler and a user. But they don't even have to do that. Blocking the entire AWS ip range isn't hard either. I doubt any craigslist users browse the site through AWS :)
  - aaronsw 19y ago
    So the issue is: 1) they were putting ads on top of Craigslist, and 2) they were using up too much bandwidth. I expect if you actually wanted to build something useful you could just email Craig and he'd get you a dump.
- aquateen 19y ago
  One example, I used to want to search for craigslist items regardless of location. I found some sites that briefly offered this service, but they were requested to stop by CL.
deleted 19y ago
[deleted]
utnick 19y ago
wonder why he does that..... cl doesnt make money off page impressions.... seems like closing the system is just hurting himself

by gscott 19y ago

optimal 19y ago

"Craigslist doesn't have an open system. Craigslist blocks ip's of those accessing there system through a web application and they send out lots of cease and desist notices" Screw it. Right now, a group here should build a newer, better Craigslist. Enough of you guys are in touch with the SV/Bay scene to bring the eyeballs required to kick it off, because as I see it, it's not a technical problem but rather a marketing one: what you need is a brief but intense "attention burst" to launch something like this. The bleeding-edge users are all here, or only one or two degrees removed, and could make it work. [Don't look at me, I'm stuck on the East Coast.]

mlinsey 19y ago

"Paypal did for eBay and it worked out for them", which is true but it should also be noted that it worked only after eBay launched a competing product that PayPal was able to beat despite the enormous distribution advantages that an eBay-run competitor had.

aaronsw 19y ago

That's a pretty serious claim; any pointers? If you've got some I'll be sure to nag Craig about it next time I see him.

fragmede 19y ago
listpic? http://mashable.com/2007/06/08/listpic-craigslist/ http://mashable.com/2007/06/08/listpic-craigslist/
- vesterr 19y ago
  Ha, with EC2 you can get around that now. Separate your website IP from your crawling IP, and every time CL blocks an IP, switch to a new one. Eventually they'll have to block the entire AWS range, but that's okay, you can crawl over cablemodem/DSL connections that use DHCP. What are they going to do, block Verizon, Comcast and Time Warner? Then you can get around referer [sic] blocks for your links to CL that the user clicks on by using a redirect, I think. You can slurp the entirety of CL daily without causing them traffic problems. I mean it's equivalent to each page getting one page view per day, which is nothing. Just keep track of URLs so you only slurp new content, and serve thumbnails off your own hosts (it's fair use).
  - nuggien 19y ago
    Why would you go to all that effort if they don't want to play with you in the first place? If they really wanted to block you, they'll do it through their lawyers.
    - vesterr 19y ago
      That stuff is really easy. And you may convince them to join you after all, like YouTube and the media companies they parasite off of.
      - nuggien 19y ago
        I didn't say it was hard. It's also very easy to differentiate between a crawler and a user. But they don't even have to do that. Blocking the entire AWS ip range isn't hard either. I doubt any craigslist users browse the site through AWS :)
- aaronsw 19y ago
  So the issue is: 1) they were putting ads on top of Craigslist, and 2) they were using up too much bandwidth. I expect if you actually wanted to build something useful you could just email Craig and he'd get you a dump.
aquateen 19y ago
One example, I used to want to search for craigslist items regardless of location. I found some sites that briefly offered this service, but they were requested to stop by CL.

deleted 19y ago

[deleted]

utnick 19y ago

wonder why he does that..... cl doesnt make money off page impressions.... seems like closing the system is just hurting himself