3 ms·
Anyone going to systematize this and create a curated list of vetted apartments for a fee? Maybe something like $9 for a month of access to one geographic area.
by jey 16y ago
Anyone going to systematize this and create a curated list of vetted apartments for a fee? Maybe something like $9 for a month of access to one geographic area.
- ericd 16y agoI've considered it briefly for PadMapper (http://www.padmapper.com http://www.padmapper.com). It would be very expensive to do for all cities, though - the volumes of listings they would have to look through are pretty enormous if you were doing it generally. One of the big things you get out of this is that you tell the person exactly what you want, and they throw away everything else. There might be some way to make it work, though.
- acgourley 16y agoPadMapper really is a good way to find listings on CL, if anyone needs it. Was my go-to for finding places in SF last year.
- potatolicious 16y agoI used PadMapper to find my new place a few months ago. Thanks again for all your work on it.
- ericd 16y agoYou're welcome, if you have any feedback, I'd love to hear it.
- jey 16y agoYou wouldn't do it for all geographic areas, just targeted ones that have sufficient demand to make it worthwhile.
- keeptrying 16y agoPadmapper rocks! Thank you for all your work on it.
- mistermann 16y agoVery cool website, nice work. Question: how do you get the data from craigslist...is there an official api,or do you scrape the site (and if so wouldn't you eventually get blocked)?
- ericd 16y agoNah, no official API, just parsing the site. It's possible that they'd choose to block it, though I've gone out of my way to give them no reason to. PadMapper's CL crawler is built to be as light as possible on their servers, and the crawling is constrained to a very small subset of the site. If it gets blocked, PM has other sources, but it would definitely suck. If you tried to make something that crawled the entire site, I would imagine that they would block you pretty quickly, though, if only for the load it would cause.
- mistermann 16y agoAny particular libraries or techniques you used for the crawling / parsing, or just coded up something from scratch for each site you crawl? (I think I noticed you even do kijiji.)
- ericd 16y agoAll of it is from scratch. Yep, I do Kijiji, but only in Canada, because I was told it's more important than CL in some parts there.