14 ms·

Bringing Craigslist Back

309 points by ericd 14y ago

SlyShy 14y ago
As someone apartment hunting in NYC I appreciate this. If I wasn't my feelings would be pretty mixed.
- ericd 14y ago
  Yeah, I went back and forth on it quite a lot. That said, the law protects this sort of thing (facts can't be copyrighted, and that's all PadMapper is showing from CL), and it seems like a pretty large net loss for the world if this isn't continued. But I'm obviously biased.
  - unreal37 14y ago
    Facts can't be copyrighted. But how is an advertisement a "fact"? Can I create a website of all the Coca-cola advertisements over the years, and call them facts? Ads aren't facts. You're still intentionally copying the CL website without the right to do that. Good luck to you. Edited: Not trying to start a "copyright isn't theft" debate.
    - Produce 14y ago
      Copying is not stealing! Christ, the music and movie industries have really fucked up the meaning of that word in recent years.
      - deleted 14y ago
        [deleted]
      - jmduke 14y ago
        Craigslist's #1 asset is their gigantic amount of classifieds. There's really nothing else going for them. Any service competing with Craigslist that uses those classifieds might not be stealing them, but its certainly in Craigslist's best interest to keep their most valuable asset safe.
        Produce 14y ago
        I'm still technically correct, which is the best kind of correct.
        jmduke 14y ago
        I wasn't saying you were incorrect, mainly because I'm not nearly well-versed in law enough to define 'stealing' in such a nuanced context. I'm simply saying that Craigslist has a vested interest in keeping their data/facts/ads/whatever on their platform.
    - jonknee 14y ago
      No, but if you created a website that showed what prices stores were selling coke for, Coke couldn't do anything. The facts of an ad are not copyrightable.
      - unreal37 14y ago
        If you went and did the work to get those prices (called or visited all of the stores in a city, for instance), then you'd be correct. But if you're scraping someone else's website in an automated fashion, and that's provably true, you're entering into the copyrightability of databases. There's a material difference - collecting data yourself versus copying someone else's collection of data. As another example, can I go to Amazon.com (without using their API), scrape all the product names and prices off their web site and map the dates and times they change? I would think not. Those too are facts, but you're basically just copying someone else's database. Can I scrape stock data off of Yahoo finance without having to pay the license fee associated with getting historical stock data?
        jonknee 14y ago
        > As another example, can I go to Amazon.com (without using their API), scrape all the product names and prices off their web site and map the dates and times they change? I would think not. Those too are facts, but you're basically just copying someone else's database. Sure. There are plenty of services that do this, it's quite handy. Here's the one I use: http://camelcamelcamel.com/ http://camelcamelcamel.com/ > Can I scrape stock data off of Yahoo finance without having to pay the license fee associated with getting historical stock data? Sure. They even provide a CSV format to make it easier for you! http://ichart.finance.yahoo.com/table.csv?s=YHOO&d=0&e=28&f=2010&g=d&a=3&b=12&c=1996&ignore=.csv http://ichart.finance.yahoo.com/table.csv?s=YHOO&d=0&#38...
  - waxpancake 14y ago
    A unique database of facts absolutely can be copyrighted. The Billboard charts and SoundScan are a great example, upheld in court. More here: http://www.bitlaw.com/copyright/database.html http://www.bitlaw.com/copyright/database.html
    - qohen 14y ago
      The Supreme Court apparently ruled overwise on this case, as discussed above, i.e. here: http://news.ycombinator.com/item?id=4219085 http://news.ycombinator.com/item?id=4219085
csmajorfive 14y ago
So does 3taps gather data with humans or what?
- ericd 14y ago
  My understanding is that they use the Google cache.
- keltex 14y ago
  "Any copying, aggregation, display, distribution, performance or derivative use of craigslist or any content posted on craigslist whether done directly or through intermediaries (including but not limited to by means of spiders, robots, crawlers, scrapers, framing, iframes or RSS feeds) is prohibited" I think this covers Google cache or any other method of gathering data. You're still in violation of their Terms of Use even if you find a clever means of getting around it.
  - eli 14y ago
    That document begins with "By accessing or using craigslist..." I imagine the argument would be that since you are not accessing or using craigslist directly you are not bound by those terms.
  - nicholasreed 14y ago
    Simply because it exists in the ToS doesn't mean it is legally enforceable. Hopefully the "legally kosher" claim means he had a lawyer look over everything, and conclude that clause in the ToS is for the courts to decide.
  - whichdan 14y ago
    Technically, wouldn't Google Cache be in violation, and therefore 3Taps & PadMapper by extension?
kefs 14y ago
The last few places I've acquired we're through padmapper aggregation of CL posts, so when PM got the C&D, I felt compelled to email CraigsList asking them to implement a fee-based API as it would be a win-win-win for everyone involved. Sadly, no reply.
heyrhett 14y ago
Can 3taps get me the content of the NY Times website via indirect means, so I can publish it on my website? I did a back of the envelope calculation that a lot of people are losing time by having to go somewhere else to get that content, and it's pissing me off.
- debacle 14y ago
  They can, but you didn't say 'fuck' enough.
  - ericd 14y ago
    You're right, I shouldn't curse/write when angry, edited.
- seanieb 14y ago
  Only if the NY Times publishes their content in a fashion that makes it hard to read, undiscovered and freakishly difficult to use in any useful manner, and your site makes it readily discoverable and very useful.
  - mkr-hn 14y ago
    As a content creator, I would prefer you bring the issues to my attention, not subvert my protections under copyright law. You don't get to do as you wish with my creations just because you don't like how they're displayed. It doesn't matter whether we're talking about the NYT or some random blog.
  - deleted 14y ago
    [deleted]
- ericd 14y ago
  It's obviously not OK to post the content of the stories on your site, but you're totally within your fair use rights to write something ABOUT their stories.
  - 14y ago
jmduke 14y ago
As a satisfied padmapper user, I'm glad to see this, but somewhat troubled at the same time. The original hubbub about Craigslist pulling padmapper's access, regardless of your stance on Craigslist's actions, was to me a cautionary tale about building apps around external APIs. (See the Twitter drama of the past weeks.) What happens next if Craigslist does the same thing to 3taps?
hornbaker 14y ago
Bold move. Curious if CL's next step will be to add noarchive tags to prevent google cache content from being crawled.
- moron 14y ago
  I can't see why they wouldn't do that. It's not like the Google cache offers much utility for Craigslist posts to begin with. Anyway, it doesn't seem right for Padmapper to be doing this, but for god's sake, could people browsing the primary source for residential ads please have a way of doing that with a map?! That's really all anyone is asking for here. Sheesh.
  - koenigdavidmj 14y ago
    > I can't see why they wouldn't do that. It's not like the Google cache offers much utility for Craigslist posts to begin with. Then there's no harm in their putting the tags in.
    - moron 14y ago
      Uh, okay?
mmanfrin 14y ago
I have a feeling there will be another C+D soon, but since I'm looking for an apartment at this moment, great. I am annoyed that CL hasn't uttered a single peep about all this. They killed a great app, and didn't even respond to the outcry. A simple 'we don't want to support competitors' would have sufficed, but their silence damns them.
- elsewhen 14y ago
  Padmapper already received a C&D... IANAL, but i think a lawsuit may be the next step.
  - grabeh 14y ago
    If the circumstances have changed, the appropriate course of action would be a further cease & desist. This would allow Padmapper the opportunity to desist from their activity which is always the right course of action before instituting proceedings.
    - elsewhen 14y ago
      The C&D mentions not just accessing CL content, but republishing it. That the content was transmitted through third parties is irrelevant for this copyright claim.
      - grabeh 14y ago
        It may be irrelevant, however you will get penalised on costs (at least in the UK) if you do not attempt to resolve the dispute prior to issuance of proceedings. I personally wouldn't want to risk the ire of a judge in issuing proceedings when at least on the face of it, PM complied with at least part of the initial cease & desist sent by CL.
      - biot 14y ago
        As others have pointed out, it's not Craigslist's content. According to CL's own terms, users own all content posted on the site. So CL has no copyright claim on it.
phillmv 14y ago
>I’ve been wrestling with whether to bring back Craigslist listings in the search results. I’ve found a way to include them that I’m told is legally kosher since it doesn’t touch their servers at all, but it still seems somewhat dickish to go against their wishes in this, and I’ve always had a lot of respect for what they’ve done for the world. Also, who wants to waste their time in court? >But then I did some back of the envelope estimates of how much of people’s time and effort it would waste if I didn’t, and it became clear how much less nice it is to waste the time of millions of apartment hunters out of stubbornness or some clearly inaccurate assumption about the will of the community. Christ, what an asshole. Translation: "We found a loophole that lets us get around the spirit of what was communicated to us so we could continue to build out our product. I'm going to conveniently step over the moral grey lines of using someone else's data without their consent by claiming my service is better."
- ericd 14y ago
  Ouch, sorry, I thought it was the right thing to do, on the balance. Like you said, it's a really morally gray decision, either way, but I think it's actually worse to do nothing.
  - sandieman 14y ago
    It was the right thing.
  - dinedal 14y ago
    Ignore this guy, you're doing the right thing, and I thank you. e: To expand, CL obviously picked on you specifically, and it deprives all the apartment hunters of your service when applied to the largest apartment listing available. I could see it if they somehow filled the void with a comparable service, before they turned it off, but by depriving padmapper access, it makes it worse for everyone.
    - tptacek 14y ago
      Be careful reading these comments. There's an overwhelming bias on HN in favor of "rules that impede startups are there to be broken", and, of course, 3/4 of the site is going to cheer you on. Remember when Mark Pincus said early stage startups need (as I remember) to lie, cheat, and steal to get off the ground? HN cheered that on too. Just remember that every nerd that has ever broken any rule --- from seeding a torrent of a movie to scraping a competing site to (yes, go look it up) exploit SQL injection to dump a company's mail spool to the Internet --- has come up with a personally compelling rationale to do it. MAFIAA, "we're helping apartment hunters", "STRATFOR was an evil intelligence company". These are all of a kind. The fact is, you compete with Craigslist. However much "better" your service is, customers don't agree: they overwhelmingly prefer Craigslist, because that's where the market is: the "killer feature" in your space is the market. So, to stay viable in the face of competition that has you beaten dead to rights, you've rationalized cheating them. I don't find what you're doing detestable, or anything like that, but it sure is annoying to watch you pat yourself on the back for it. I'm with Phillip. Have you read a lot of his comments? I have. He's not dumb, nor is he a troll.
mootothemax 14y ago
Can someone explain to me how this works legally? It seems a bit too convenient that you can ignore someone's T&Cs and use an unlicensed third-party service instead.
- eli 14y ago
  I don't think there's a whole lot of caselaw here, so I think it's all a bit of a grey area. One thing I can say from experience is that judges typically frown on "clever hacks" to get around legal obligations. IANAL.
  - mindslight 14y ago
    A TOS is only a legal obligation if one enters into said agreement in the first place, which is questionable even when one has signed up for an account. In most cases, it's mere wishful thinking or just extra garbage for lawlyers to bully people with.
- debacle 14y ago
  The meatspace analogy would be having 100 data technicians transcribe all of the Craigslist ads. You're still stealing the Craigslist content and database, ergo you're still liable for damages.
  - delackner 14y ago
    There is no protection for a database of public facts under US law, contrary to say the UK. Craiglist's data is all public facts.
    - debacle 14y ago
      Posted elsewhere in this thread: http://www.bitlaw.com/copyright/database.html http://www.bitlaw.com/copyright/database.html http://www.bitlaw.com/source/cases/copyright/feist.html http://www.bitlaw.com/source/cases/copyright/feist.html If you read the second link, you'll see that it is exactly what the OP is doing and it has already been decided in favor of the database owner in case law.
colinsidoti 14y ago
I can't help but think you played this poorly. It sucks CL isn't going to let you get their data, but I really don't think this workaround is going to change anything (although kudos for the neat Google cache hack from 3taps). I imagine your development efforts would be better spent by leveraging the small but dedicated following you clearly have. There must be ways to take advantage of it without opening yourself up to more litigation.
- ericd 14y ago
  You may be right, I'm flying by the seat of my pants here (after having talked to some lawyers). My goal isn't to supplant Craigslist, which is what you seem to be suggesting - that's far too big a task to take on, even with my sites' substantial usage, and not really something I'm personally interested in - I think that classifieds are a mostly-solved problem, with the exception of reputation and security. I think CL can get there, they seem to be hiring devs.
  - colinsidoti 14y ago
    Yea it sucks your dealing with CL and not Facebook or you probably would have been acquired. I say try and move the traffic to pad. It's certainly a big task, but not "far too big". Pick the region where you have the most significant traffic and start there. I haven't looked deeply into your site at all, but I know you have Garry Tans support, ask him what you need to do to get in YC. Get "Craigslist is the Killer" as a headline on TechCrunch. Piggyback the Padlister browser add-on with other common downloads, then push a popup to users posting apartment listings on craigslist saying "Also list on Padmapper?" There are all sorts of things you can try. There are hundreds of people on HN that wish they had the traffic you have. Use that momentum before it dies off. http://siteanalytics.compete.com/padmapper.com/ http://siteanalytics.compete.com/padmapper.com/
    - bluesnowmonkey 14y ago
      I think if you look up "network effect" in the dictionary, it's going to talk about Craigslist. Good luck overcoming that. A great strategy here would have been to establish a strong relationship with the people at Craigslist from the beginning. No idea if that was tried or not. Seriously, was the C&D letter the first communication exchanged between Padmapper and CL? What a missed opportunity. I wonder if this entire disaster could have been averted by calling the CL guys up early on and taking them out to lunch. Make friends, send them Christmas cards, then when this disagreement comes up you handle it over a round of golf instead of a C&D letter.
grabeh 14y ago
There's obviously a big fat indemnity clause in 3Taps' terms which means you're on the hook for claims made by third parties against 3Taps for loss arising as a result of the usage of the service. Having said that if CL provides access to 3Taps on terms which allow third parties to take data from 3Taps, then this may be permissible. It of course depends on the terms of 3Taps' access though (I admit to not being completely familiar with the ins and outs of 3Taps' service though). The problem of course is that CL have already shown a dislike of Padmapper's methods and as a result this workaround is unlikely to meet with CL's approval. Common sense would suffice to tell you that. Having said that, it's a real shame CL aren't open to discussions over licensing data (maybe they are of course but the price is too high).
- ericd 14y ago
  Nah, they weren't open to discussing any sort of deal involving the website (they were willing to license data for the mobile app).
  - stcredzero 14y ago
    Why didn't you go that route? There are so many phones and tablets out there now, you'd have accomplished almost as much public good with far less controversy.
    - ericd 14y ago
      Good question. Basically, it's not really the best way for people to browse this kind of stuff, and there are still lots of people without phones/tablets. You can get a much better picture of things with PM maximized on a 20" screen than you can with a 4" one. The phone is more of a backup for when you're on the road or in a pinch.
      - stcredzero 14y ago
        The phone is more of a backup for when you're on the road or in a pinch. My girlfriend is looking for an apartment in Houston, and sometimes browses Craiglist with her iPad.
glesica 14y ago
I don't understand how CL can even exercise this kind of control. I'm not a lawyer, of course, but it seems reasonable to me that apartment listings would be considered "facts", just like baseball statistics. So then the CL terms of use are the only obstacle, but can you really be held to those if you didn't read them and weren't somehow forced/encouraged to do so? Perhaps so, but then I would find that a little scary. Just seems like a bad situation all around, but in my mind Padmapper has the moral (though perhaps not legal, again, not a lawyer) high ground, at least relatively speaking.
- angryasian 14y ago
  no its very simple. Don't submit your listing to craigslist them. Submit it to a site that is indexable friendly.
- therandomguy 14y ago
  Let me try. What you say is correct. The listing is "fact". And CL is not preventing posting this fact to other services. The user can post the same listing to CL, Rent.com and dozen other sites. What they are controlling is access to their servers.
Spooky23 14y ago
My question is: Why are people using craigslist? Fortunately for me, I purchased a home before all apartment ads migrated to CL. For the life of me, I don't get why it's so popular. The site is awful in at least a half dozen different ways.
- dinedal 14y ago
  It's free and easily accessible. Compare to the costs involved in getting an ad listed in a dead tree.
- pavel_lishin 14y ago
  > Why are people using craigslist? Because everyone else is.
- sharkweek 14y ago
  I feel like they snagged the community of online classifieds first and have been able to hold on pretty tightly, as people like my dad love that almost nothing has changed.
- smackfu 14y ago
  Ads migrated there because it was free, and a lot more agile than newspaper ads (wait a week for my ad to go live??). They stay because of network effects. And because it's good enough.
- brittohalloran 14y ago
  The funny thing is -- PadMapper and 3Taps and the like are actually perpetuating the usefulness of Craigslist: "everyone scrapes Craigslist, so if you just post it on Craigslist it will be pulled into all those other services automatically"
- mrgoldenbrown 14y ago
  Yes, the site is awful. But it is less awful than the alternatives. Therefore, people use it. Please, on behalf of all apartment hunters everywhere, (but especially in high tech/high rent places like the Bay area or Boston), tell us what the better option is! I say this in all seriousness. The idea that PadMapper was saving vast amount of man hours was not an exaggeration. I wasted many hours clicking on craigslist ads to find my current apartment.
ilamont 14y ago
The founder of 3Taps posted this on Quora last year, in response to a question about disrupting Craigslist: ... the postings in question are public facts about exchanges. Just as any price/supply/demand in a marketplace is open for any and all to notate and republish, so too is the entire set of Craigslist data -- as these offers between seekers and providers are clearly in the public domain. Historically, Craigslist has attempted to block access by others to the comprehensive use of this data. They block many 3rd parties who try to gain access to the data, and sometimes threaten to sue and bankrupt others as if they themselves created the underlying data and hold copyright like property rights over the same. But public facts are public property. And while some think that predatory Terms of Use demanding that you hand over the Brooklyn Bridge in liquidated damages if you don't comply with some obscure (and potentially constitutionally void) constraint will stand in court -- such absurdities will break if exposed. A fear, uncertainty,dread approach over access to data breaks down in a world where Google already indexes all of Craigslist data and caches that information all over the internet (for search performance results). If its possible and legal for Google, then why not for any and everyone else to also index and offer access to the same data. In short, Google doesn't get special secondary property rights to privatize public data to the exclusion of anyone else. Equal access to exchange data and search data is a principle in parallel to the notions of net neutrality. The points above are not a theoretical discourse. Look at 3taps.com/developers to see the execution of this concept. And look at what a 3rd party application (craiggers.com) can do in recreating the whole of Craigslist in a format that gives access to data in a way that is not remotely possible in the legacy Craigslist offering. Craiggers is a perfect example that the function of displaying Craigslist data (rather than gathering it) is a totally distinct (and competitive) marketplace, even if there are still huge network effects in the gathering of Craigslist postings. Note, Craiggers does NOT disrupt the existing Craigslist revenue model for Craigslist. It simply opens up the field (along with any other developer building on 3taps assisted access to Craigslist data) that wants to build on top of (rather than compete with) the network effects of Craigslist. Think Kayak and Indeed, but now for the whole body of data covered by Craigslist accessible, rather than just a single vertical. http://www.quora.com/Craigslist/Why-hasnt-another-product-disrupted-and-replaced-Craigslist/answer/Greg-Kidd http://www.quora.com/Craigslist/Why-hasnt-another-product-di...
- 14y ago
IanMechura 14y ago
I see some very interesting comments on this thread. I remember not long ago lots of people where upset that people where scraping stackoverflow and getting a higher ranking in google results. This was a threat to S/O's business model model and it made lots of people on this board uneasy. How is Pad mapper any different? How does the fact that this site has a better UI design somehow give them a pass to use data a company has spent millions of dollars gathering and feels is being used in a way that is a threat to their business model?
- delackner 14y ago
  Spent millions gathering? Two basic problems with that mindset. First, spending money to assemble a database of public facts does not magically give you a right to protect your investment in an uncopyrightable asset. Second, I don't know what millions you are talking about since craiglist's only asset is the users, and the users are just a result of tremendous network effects. They had a great idea and got very lucky, but gathering their massive userbase had nothing to do with spending money.
  - delackner 14y ago
    Could someone explain what they disagree with? Do you disagree that databases of public facts are not copyrightable? With the idea that craigslist's investment in their system is irrelevant compared to the value of their userbase? With the idea that craigslist's main success is just being lucky?
- base698 14y ago
  Why would you even need to scrape it? It's available as a torrent: http://blog.stackoverflow.com/2009/06/stack-overflow-creative-commons-data-dump/ http://blog.stackoverflow.com/2009/06/stack-overflow-creativ...
stcredzero 14y ago
After reading the post and discussion, my conclusion is that Craigslist is making a policy mistake. They should make the same calculation of public good that ericd made and license 3rd party websites that add value to Craigslist, provided the 3rd parties do not harm the Craigslist infrastructure or business models. Doing otherwise is just holding up progress, and is a losing game in the long run. Facilitating value to users would be a win for them, as it was in the past.
- debacle 14y ago
  Creating a site where buyers can browse property listings does harm the Craigslist business model. To say otherwise expresses a lack of understanding of online ecosystems.
  - stcredzero 14y ago
    As far as I can tell, they aren't posting the full listing. They are only aggregating the material facts about the listing. As far as I can tell, short of actually hoofing it to the place, you still have to get on Craigslist to make contact with the landlord.
    - hollerith 14y ago
      What Craigslist is afraid of (probably) is Padmapper becoming the first place most people turn to when looking for an apartment. Once that happens, Padmapper can cut Craigslist out of the loop by inviting landlords to make Padmapper listings instead of Craigslist listings. (Note that once Padmapper become to "go to" place for apartment hunters, Padmapper's refusing to continue to host or "index" Craigslist listings helps towards that goal.) Regardless the morality or legality of Padmapper's latest action, it is pretty clear to those of us who have spent a lot of time observing online ecosystems and software ecosystems that Padmapper's latest action is not in the interests of Craigslist since (even if Craigslist is not out to make money, but rather out to influence the world for the better) the only important source of Craigslist's potential for making money and of Craigslist's influence is Craigslist's being the first place people turn to when apartment hunting and doing other important things. That is why Craigslist is against Padmapper's latest action even if Padmapper never posts the full Craigslist listing.
dpcan 14y ago
I think everyone should just step back and look at the big picture. This guy is finding any means necessary to add Craiglist data back into his application and Craigslist does not want him to. I think the "gray" area is pretty black and white. Craigslist said no. He's doing it anyway - even if indirectly. Here's how I see it: Me: Don't go in my house. You: But you left the back door unlocked by accident, so I thought I'd just come in and eat your food. In the real world this would not be acceptable. Even though we are talking apples, bananas, and oranges, the same ethical perspective should be employed. Craig said no, so we respect his wishes. It's the RIGHT thing to do. If he adds an API someday, then he's sharing the key to his house with us, until then. Doors are locked, too bad, move on.
- natrius 14y ago
  It it invalid to apply analogies involving physical property to intellectual property. The two are not similar enough for that to be a convincing argument, especially since mere facts aren't protected intellectual property at all. http://en.wikipedia.org/wiki/Feist_v._Rural http://en.wikipedia.org/wiki/Feist_v._Rural
  - dpcan 14y ago
    This seems to always be the reply when analogies come up, which is why I said it was apples, bananas, and oranges. The point being, what's mine is mine, and if I say you can't have it, I want you to respect my wishes. Sure you can probably justify endlessly, or interpret differently, or stretch the truth, or bend the rules, but one business is saying, don't use our data this way, and regardless of this, the other business is finding a way to use the data. Seems wrong to me. EDIT: Because I can't reply deeper to these threads, I have to edit here. The "mine" above is not intended to be in regards to the information or content on Craigslist. I'm not saying they own it. I think there is a relationship between the person who posted the information to Craigslist and Craigslist itself. I don't think anyone has a right to become some part of that relationship without both sides agreeing. Maybe it's because I'm a freelancer. If I have a relationship with a Client to build them something, and someone else decides, of their own accord, that they are now part of that relationship, I have the right to tell them to back off, and I believe they should respect my wishes. I'm not talking data ownership. I'm not talking about legal matters. I'm saying, human to human, they should just be cool with Craigslist wishes and go another way.
tubbo 14y ago
Dear PadMapper, We hate your app and you can't use our API. Sincerely, Craigslist + + + Dear Cragislist, Fuck you. Love, PadMapper
rdl 14y ago
Thank you for doing this. My only problem with it is that you're rewarding craigslist for being obnoxious tools, but ultimately it's better for the users, so that's an acceptable sacrifice. 3taps is pretty awesome. I wish they'd do an AMA or something.
raganwald 14y ago
Cartographers, encyclopedicians, and dictionary aggregators have always included some false data as a honeypot: An alleyway might not really exist. An obscure word might have been made up. A famous person might be entirely fictitious. The goal is to catch people copying their books. If you use “original research,” you’d never say that Braythwayt Street runs parallel to Rhodes Avenue in Toronto, you must have copied our map data without permission. I’m curious as to how padmapper intendeds to solve this problem. If Craigslist seed their listings with something false, that something is entirely craigslist’s copyright. It isn’t a “fact.” They don’t have a right to reproduce it. Lawyers, please chime in.
- keithpeter 14y ago
  Oh, short story idea there. Someone claims to live in the made up address and generates an identity complete with utility bills... Neal S here we come...
  - raganwald 14y ago
    Fred Saberhagen used the concept of a map bunny as a plot device in one of his Berserker short stories.
- mikeryan 14y ago
  I believe padmapper is kind of relying on 3taps to take the hit. 3taps apparently is DMCA compliant. Long story short I'd expect 3taps to be craigslist legal target. (though actually I guess 3taps doesn't actually use the data so I'm not sure if what they're doing is technically illegal until padmapper uses it - I'm going to stop armchair lawyering now)
  - deleted 14y ago
    [deleted]
  - dpark 14y ago
    I'm not sure it matters what 3taps says or whether they are DMCA compliant. If I host pirated movies on my server and post in big bold letters, "I promise that you can redistribute them. It's totally legal and I'm responsible for any legal actions.", you're probably still getting sued by the MPAA if you redistribute the movies, and they'll probably win. I'm not saying Craigslist has any legitimate claim over the data, merely that if they do, accessing it via an indirect means probably doesn't change anything legally.
eli 14y ago
It's funny how many people are demanding that Craigslist sell out its user-generated content. I understand why people want this, but isn't CL being incredibly respectful of its users by allowing their data to be used only for the specific purpose for which it was posted and that's it?
- crazygringo 14y ago
  I don't think so. Users are on Craigslist to buy/sell, because it's the biggest place for that. If you post to CL, virtually everyone wants the largest audience possible, so 99.9% of people will be thrilled to have their post copied to other services. The only annoying thing would be if you take down your post from CL after you've bought/sold your thing, and you continue to get calls/emails from people getting it via copied data. But sites that copy just need to stay up-to-date on whether postings are still up or not...
seagreen 14y ago
What about a service that lets you post to every barter site at the same time, not just craigslist. Has that been tried? If you could get people to use it (through better UI and more exposure for what they're selling) you could break craigslist's hold on the marketplace.
duked 14y ago
Really, really stupid. I mean pretending you want to help people find apartment to justify you're action is ridiculous. You're running a business, be honest about it. Now craigislist do what they want with their data, you are just trying to build a service that "adds value" on top of someone else infrastructure( I mean "add" because I am one of the few who actually like the way cragislist display posts no bloaty js, no ugly pics no maps ... fast and simple).
- JackC 14y ago
  I find it difficult to believe that you have searched for an apartment in a tough rental market on Craigslist and consider their interface "fast and simple." If you've never had the task of finding an apartment within a mile of a given spot in a big metro area, you're not looking for what PadMapper offers. If you have had that problem, and you've never sworn out loud at Craigslist, you're a saint.
  - duked 14y ago
    I was looking for an apartment 6 month ago in Reston, VA and I used craigslist for that purpose. I looked for a couple of zipcode (that I heard were good, since I was just moving in the US at that time) and honestly it worked great. Wouldn't it be the same in a big metro area ? just looking for interesting zipcodes ?
    - drstewart 14y ago
      You realize in some cities a few blocks makes a GIANT difference in neighborhood, right? Same zip code. Being able to visually filter those out is a major time saver.
      - JackC 14y ago
        The other issue is where I live, people put all kinds of random keywords in their Craigslist apartment ads to try to get attention. "Just 25 minutes' walk to Foo Square, Bar Square, and Baz Square! Four hours by horse to Lexington, and a short plane ride to London!" If I want to know what's within a 10-minute walk of a given location, I basically have to read every listing with my town's name in it every day and see if they included an address. Or spend 5 seconds glancing at PadMapper.
bambax 14y ago
How do 3taps do it? Some comments say they "use Google cache", which doesn't explain much: - Google is notorious for fighting bots pretty aggressively, how do 3taps avoid this? - even if 3taps found a special access to Google cache, it would be trivial for CL to prevent Google or anyone else from caching its listings, thus cutting off the rest of the food chain that feeds on caches; why don't they do it? Is it possible that 3taps use another method, such as "manually" copy-pasting listings with Mechanical Turk?
Zakuzaa 14y ago
http://traffic.alexa.com/graph?w=400&h=220&o=f&c=1&y=t&b=ffffff&n=666666&r=3m&u=padmapper.com&#38 http://traffic.alexa.com/graph?w=400&h=220&o=f&c...; Traffic took a great nosedive from the day they announced "bye bye" to CL.
- huckleberries21 14y ago
  By your own metric, so did traffic for CL, and CL is the 8th most visited site in the US, so their Alexa traffic data is probably more stable than traffic data for PadMapper. http://traffic.alexa.com/graph?w=400&h=220&o=f&c=1&y=r&b=ffffff&n=666666&r=3m&u=craigslist.org&#38 http://traffic.alexa.com/graph?w=400&h=220&o=f&c...;
rjsamson 14y ago
Not sure if this has been asked, but a philosophical question for Eric: if someone decided to start a padmapper competitor tomorrow, would you be okay with them using your data for free?
- huckleberries21 14y ago
  I don't see why not. By being a competitor that scrapes PM, they are accepting that there will be a delay in their data versus when it's available on PM. So unless their site was significantly better than PM in some way, in the really competitive apartment hunting markets, they would be strictly inferior to PM.
rprasad 14y ago
Consider, if the situation was reversed, and Craiglist was taking Padmapper's data without permission, would everyone on HN would be supporting Padmapper attempting to block Craiglist from reposting its listings? In such a scenario, Craiglist reposting the Padmapper listings would be a net gain for user's and posters (even though the posters never gave Craiglist such permission) because the listings would be exposed to an exponentially larger viewership. Being a startup is not a license (pun intended) to ignore other people's rights. Padmapper is in the wrong here, and if they proceed down this path they could face a ruinous (and fully justified) lawsuit.
lancefisher 14y ago
I appreciate that Eric is doing this in the open. The moral argument could go either way, but he is letting everyone know what he is doing and why. He deserves credit for inviting feedback on it. Personally, I hope padmapper does well.
jgh 14y ago
Padmapper is awesome, way way better than using Craiglist to apartment hunt, even if the data ultimately comes from Craigslist (among others). Maybe craigslist should buy padmapper instead of getting all lawsuity and take advantage of just how good their interface is for apartment hunting
pbreit 14y ago
For all the people mis-interpreting CL's TOUs: "You automatically grant and assign to CL, and you represent and warrant that you have the right to grant and assign to CL, a perpetual, irrevocable, unlimited, fully paid, fully sub-licensable (through multiple tiers), worldwide license to copy, perform, display, distribute, prepare derivative works from (including, without limitation, incorporating into other works) and otherwise use any content that you post. You also expressly grant and assign to CL all rights and causes of action to prohibit and enforce against any unauthorized copying, performance, display, distribution, use or exploitation of, or creation of derivative works from, any content that you post (including but not limited to any unauthorized downloading, extraction, harvesting, collection or aggregation of content that you post)."
etrain 14y ago
"Certainly by the time you read this." Code isn't live yet, at least for me, ericd - is this conversation giving you pause about deploying the new feature, or is it standard ops issues? Also, this couldn't come at a better time for me. I love padmapper and am moving to the bay area in 3 weeks, I need this, man!
- ericd 14y ago
  Sorry, was having some sysops issues, thanks for the heads up.
dredmorbius 14y ago
Eric's confusing method with goal. Craigslist's past method was to claim TOU violation on the part of PadMapper, denying PM ready use of its content. Its goal is to maintain market position as the dominant classifieds listing service. TOU is just one way for CL to enforce its market position, and as a commercial entity, it's very likely it will continue to do so. There's very little reason CL couldn't create an alternate legal claim based on content (the cartographic watermarking example comes to mind) to make multiple infringement claims against PM, which would invoke a whole host of copyright remedies ranging from injunctions and monetary damages to DMCA takedown notices (which could comprise the entire service depending on how construed by PM's hosting provider(s)). There's also the notion of a compilation copyright, in which a collection of CL postings could have standing. While Feist v. Rural Electric holds that a simple compilation of facts doesn't meet the minimal authorship requirements of copyright, a compilation of postings with some level of curation to CL might. I'm not aware of relevant caselaw here. My own suggestion would be that PM pursue a dual strategy of coming to a business agreement with CL (PM's interface is vastly superior) while curating its own set of rental listings independently (CL is ripe for disruption here). Disclaimer: I'm a long-time CL user and fan of much of its philosophy. I'm also a recent fan of PM. I hope the two will either work something out cooperatively, or manage to improve apartment and classifieds search by competitive means. I don't particularly care which.
Baba_Chaghaloo 14y ago
>I've always had a lot of respect for what they've done for the world Did I miss something? What has Craigslist done for the world besides put newspapers out of business?
- genwin 14y ago
  Using CL I furnished a whole house with super high-quality stuff for less than US$2K. Wouldn't have been realistic pre-CL since traditional classified ads don't have pictures. I also found my house through CL.
whyenot 14y ago
So, instead of directly scraping CL's data, they are now using an API that scrapes Google's cache of CL's data? That seems kind of sleazy.
trotsky 14y ago
I think it's perfectly fine if you want to play games dancing around what's legal and forge ahead against Craigslist's wishes. There's a long history of that kind of behavior in information services and they may or may not be able to stop you. But making a big deal out of being some kind of ux hero is pretty tacky. At the end of the day, no matter how small you are you're still a commercial enterprise gaining benefit at another commercial enterprise's expense. You're bidding on eyeballs with an offer of time savings. A vast majority of the web deliberately degrades their user experience in order to achieve business goals. Ads are almost always at least somewhat distracting, retailers stretch out checkout flows for up selling, sites "recommend" their high margin items ebay requires paypal and facebook sabotages their privacy management so you're less likely to bother. If CL was to build a bunch of sweet vertical specific apps for their sections they'd need a ton more developers, designers, testers, project managers and so on. To compensate for the greatly increased burn rate they'd need to monetize a lot better which almost always means features, content or policies that negatively impact the user. Or maybe they'd have just taken too much VC to pay for it and gone under in '02 when the bottom fell out. So by all means jockey away it's healthy for the industry. But crowing about it is a bit like taking a shit on the guy whose shoulders you're standing on while bragging about how tall you are.
- ericd 14y ago
  I'm not making a big deal about this, and I have a lot of respect for Craigslist, don't put words in my mouth. I think Craigslist is nearly perfect for the vast majority of things, I'm not on a crusade to "disrupt" them. It's just that for a long, drawn out, location-based search, it's objectively terrible, and I'm trying to patch that bug.
  - J3L2404 14y ago
    http://news.ycombinator.com/item?id=4220106 http://news.ycombinator.com/item?id=4220106
  - trotsky 14y ago
    It would seem like the definition of respect mostly conflicts with your behavior. re·spect 1. To feel or show deferential regard for; esteem. 2. to pay proper attention to; not violate "to respect Swiss neutrality" 3. to show consideration for; treat courteously or kindly
egillie 14y ago
Would a client-side padmapper be less morally gray?
declan 14y ago
I posted a look at the legal issues on Google+ here: https://plus.google.com/112961607570158342254/posts/HcCuHVG7rLM https://plus.google.com/112961607570158342254/posts/HcCuHVG7... Here's an excerpt from the post: Eric's blog post today says that "using 3taps just makes it so it’s not a TOU issue." The problem is it doesn't. Craigslist's lawyers already thought of this, which is why the TOU also prohibits the "display" and "derivative use" of "any content posted on craigslist," which PadMapper is clearly doing. As Craiglist's lawyer is likely to put it in the next C&D, assent to the TOU is still implied. [...] If 3taps is in fact scraping Google's cache, they're also violating Google's terms of use, which prohibits any effort "to access [search results] using a method other than the interface and the instructions that we provide." Anyone want to bet on how long it will be before Google blocks them through technical countermeasures, or sends its own C&D? And if 3taps is in the business of the TOU-prohibited "distribution" of Craigslist data, which it appears to be, it should expect to get a C&D from Craigslist too.

309 points by ericd 14y ago

SlyShy 14y ago

As someone apartment hunting in NYC I appreciate this. If I wasn't my feelings would be pretty mixed.

ericd 14y ago
Yeah, I went back and forth on it quite a lot. That said, the law protects this sort of thing (facts can't be copyrighted, and that's all PadMapper is showing from CL), and it seems like a pretty large net loss for the world if this isn't continued. But I'm obviously biased.
- unreal37 14y ago
  Facts can't be copyrighted. But how is an advertisement a "fact"? Can I create a website of all the Coca-cola advertisements over the years, and call them facts? Ads aren't facts. You're still intentionally copying the CL website without the right to do that. Good luck to you. Edited: Not trying to start a "copyright isn't theft" debate.
  - Produce 14y ago
    Copying is not stealing! Christ, the music and movie industries have really fucked up the meaning of that word in recent years.
    - deleted 14y ago
      [deleted]
    - jmduke 14y ago
      Craigslist's #1 asset is their gigantic amount of classifieds. There's really nothing else going for them. Any service competing with Craigslist that uses those classifieds might not be stealing them, but its certainly in Craigslist's best interest to keep their most valuable asset safe.
      - Produce 14y ago
        I'm still technically correct, which is the best kind of correct.
        jmduke 14y ago
        I wasn't saying you were incorrect, mainly because I'm not nearly well-versed in law enough to define 'stealing' in such a nuanced context. I'm simply saying that Craigslist has a vested interest in keeping their data/facts/ads/whatever on their platform.
  - jonknee 14y ago
    No, but if you created a website that showed what prices stores were selling coke for, Coke couldn't do anything. The facts of an ad are not copyrightable.
    - unreal37 14y ago
      If you went and did the work to get those prices (called or visited all of the stores in a city, for instance), then you'd be correct. But if you're scraping someone else's website in an automated fashion, and that's provably true, you're entering into the copyrightability of databases. There's a material difference - collecting data yourself versus copying someone else's collection of data. As another example, can I go to Amazon.com (without using their API), scrape all the product names and prices off their web site and map the dates and times they change? I would think not. Those too are facts, but you're basically just copying someone else's database. Can I scrape stock data off of Yahoo finance without having to pay the license fee associated with getting historical stock data?
      - jonknee 14y ago
        > As another example, can I go to Amazon.com (without using their API), scrape all the product names and prices off their web site and map the dates and times they change? I would think not. Those too are facts, but you're basically just copying someone else's database. Sure. There are plenty of services that do this, it's quite handy. Here's the one I use: http://camelcamelcamel.com/ http://camelcamelcamel.com/ > Can I scrape stock data off of Yahoo finance without having to pay the license fee associated with getting historical stock data? Sure. They even provide a CSV format to make it easier for you! http://ichart.finance.yahoo.com/table.csv?s=YHOO&d=0&e=28&f=2010&g=d&a=3&b=12&c=1996&ignore=.csv http://ichart.finance.yahoo.com/table.csv?s=YHOO&d=0&#38...
- waxpancake 14y ago
  A unique database of facts absolutely can be copyrighted. The Billboard charts and SoundScan are a great example, upheld in court. More here: http://www.bitlaw.com/copyright/database.html http://www.bitlaw.com/copyright/database.html
  - qohen 14y ago
    The Supreme Court apparently ruled overwise on this case, as discussed above, i.e. here: http://news.ycombinator.com/item?id=4219085 http://news.ycombinator.com/item?id=4219085

csmajorfive 14y ago

So does 3taps gather data with humans or what?

ericd 14y ago
My understanding is that they use the Google cache.
keltex 14y ago
"Any copying, aggregation, display, distribution, performance or derivative use of craigslist or any content posted on craigslist whether done directly or through intermediaries (including but not limited to by means of spiders, robots, crawlers, scrapers, framing, iframes or RSS feeds) is prohibited" I think this covers Google cache or any other method of gathering data. You're still in violation of their Terms of Use even if you find a clever means of getting around it.
- eli 14y ago
  That document begins with "By accessing or using craigslist..." I imagine the argument would be that since you are not accessing or using craigslist directly you are not bound by those terms.
- nicholasreed 14y ago
  Simply because it exists in the ToS doesn't mean it is legally enforceable. Hopefully the "legally kosher" claim means he had a lawyer look over everything, and conclude that clause in the ToS is for the courts to decide.
- whichdan 14y ago
  Technically, wouldn't Google Cache be in violation, and therefore 3Taps & PadMapper by extension?

kefs 14y ago

The last few places I've acquired we're through padmapper aggregation of CL posts, so when PM got the C&D, I felt compelled to email CraigsList asking them to implement a fee-based API as it would be a win-win-win for everyone involved. Sadly, no reply.

heyrhett 14y ago

Can 3taps get me the content of the NY Times website via indirect means, so I can publish it on my website? I did a back of the envelope calculation that a lot of people are losing time by having to go somewhere else to get that content, and it's pissing me off.

debacle 14y ago
They can, but you didn't say 'fuck' enough.
- ericd 14y ago
  You're right, I shouldn't curse/write when angry, edited.
seanieb 14y ago
Only if the NY Times publishes their content in a fashion that makes it hard to read, undiscovered and freakishly difficult to use in any useful manner, and your site makes it readily discoverable and very useful.
- mkr-hn 14y ago
  As a content creator, I would prefer you bring the issues to my attention, not subvert my protections under copyright law. You don't get to do as you wish with my creations just because you don't like how they're displayed. It doesn't matter whether we're talking about the NYT or some random blog.
- deleted 14y ago
  [deleted]
ericd 14y ago
It's obviously not OK to post the content of the stories on your site, but you're totally within your fair use rights to write something ABOUT their stories.
- 14y ago

jmduke 14y ago

As a satisfied padmapper user, I'm glad to see this, but somewhat troubled at the same time. The original hubbub about Craigslist pulling padmapper's access, regardless of your stance on Craigslist's actions, was to me a cautionary tale about building apps around external APIs. (See the Twitter drama of the past weeks.) What happens next if Craigslist does the same thing to 3taps?

hornbaker 14y ago

Bold move. Curious if CL's next step will be to add noarchive tags to prevent google cache content from being crawled.

moron 14y ago
I can't see why they wouldn't do that. It's not like the Google cache offers much utility for Craigslist posts to begin with. Anyway, it doesn't seem right for Padmapper to be doing this, but for god's sake, could people browsing the primary source for residential ads please have a way of doing that with a map?! That's really all anyone is asking for here. Sheesh.
- koenigdavidmj 14y ago
  > I can't see why they wouldn't do that. It's not like the Google cache offers much utility for Craigslist posts to begin with. Then there's no harm in their putting the tags in.
  - moron 14y ago
    Uh, okay?

mmanfrin 14y ago

I have a feeling there will be another C+D soon, but since I'm looking for an apartment at this moment, great. I am annoyed that CL hasn't uttered a single peep about all this. They killed a great app, and didn't even respond to the outcry. A simple 'we don't want to support competitors' would have sufficed, but their silence damns them.

elsewhen 14y ago
Padmapper already received a C&D... IANAL, but i think a lawsuit may be the next step.
- grabeh 14y ago
  If the circumstances have changed, the appropriate course of action would be a further cease & desist. This would allow Padmapper the opportunity to desist from their activity which is always the right course of action before instituting proceedings.
  - elsewhen 14y ago
    The C&D mentions not just accessing CL content, but republishing it. That the content was transmitted through third parties is irrelevant for this copyright claim.
    - grabeh 14y ago
      It may be irrelevant, however you will get penalised on costs (at least in the UK) if you do not attempt to resolve the dispute prior to issuance of proceedings. I personally wouldn't want to risk the ire of a judge in issuing proceedings when at least on the face of it, PM complied with at least part of the initial cease & desist sent by CL.
    - biot 14y ago
      As others have pointed out, it's not Craigslist's content. According to CL's own terms, users own all content posted on the site. So CL has no copyright claim on it.

phillmv 14y ago

>I’ve been wrestling with whether to bring back Craigslist listings in the search results. I’ve found a way to include them that I’m told is legally kosher since it doesn’t touch their servers at all, but it still seems somewhat dickish to go against their wishes in this, and I’ve always had a lot of respect for what they’ve done for the world. Also, who wants to waste their time in court? >But then I did some back of the envelope estimates of how much of people’s time and effort it would waste if I didn’t, and it became clear how much less nice it is to waste the time of millions of apartment hunters out of stubbornness or some clearly inaccurate assumption about the will of the community. Christ, what an asshole. Translation: "We found a loophole that lets us get around the spirit of what was communicated to us so we could continue to build out our product. I'm going to conveniently step over the moral grey lines of using someone else's data without their consent by claiming my service is better."

ericd 14y ago
Ouch, sorry, I thought it was the right thing to do, on the balance. Like you said, it's a really morally gray decision, either way, but I think it's actually worse to do nothing.
- sandieman 14y ago
  It was the right thing.
- dinedal 14y ago
  Ignore this guy, you're doing the right thing, and I thank you. e: To expand, CL obviously picked on you specifically, and it deprives all the apartment hunters of your service when applied to the largest apartment listing available. I could see it if they somehow filled the void with a comparable service, before they turned it off, but by depriving padmapper access, it makes it worse for everyone.
  - tptacek 14y ago
    Be careful reading these comments. There's an overwhelming bias on HN in favor of "rules that impede startups are there to be broken", and, of course, 3/4 of the site is going to cheer you on. Remember when Mark Pincus said early stage startups need (as I remember) to lie, cheat, and steal to get off the ground? HN cheered that on too. Just remember that every nerd that has ever broken any rule --- from seeding a torrent of a movie to scraping a competing site to (yes, go look it up) exploit SQL injection to dump a company's mail spool to the Internet --- has come up with a personally compelling rationale to do it. MAFIAA, "we're helping apartment hunters", "STRATFOR was an evil intelligence company". These are all of a kind. The fact is, you compete with Craigslist. However much "better" your service is, customers don't agree: they overwhelmingly prefer Craigslist, because that's where the market is: the "killer feature" in your space is the market. So, to stay viable in the face of competition that has you beaten dead to rights, you've rationalized cheating them. I don't find what you're doing detestable, or anything like that, but it sure is annoying to watch you pat yourself on the back for it. I'm with Phillip. Have you read a lot of his comments? I have. He's not dumb, nor is he a troll.

mootothemax 14y ago

Can someone explain to me how this works legally? It seems a bit too convenient that you can ignore someone's T&Cs and use an unlicensed third-party service instead.

eli 14y ago
I don't think there's a whole lot of caselaw here, so I think it's all a bit of a grey area. One thing I can say from experience is that judges typically frown on "clever hacks" to get around legal obligations. IANAL.
- mindslight 14y ago
  A TOS is only a legal obligation if one enters into said agreement in the first place, which is questionable even when one has signed up for an account. In most cases, it's mere wishful thinking or just extra garbage for lawlyers to bully people with.
debacle 14y ago
The meatspace analogy would be having 100 data technicians transcribe all of the Craigslist ads. You're still stealing the Craigslist content and database, ergo you're still liable for damages.
- delackner 14y ago
  There is no protection for a database of public facts under US law, contrary to say the UK. Craiglist's data is all public facts.
  - debacle 14y ago
    Posted elsewhere in this thread: http://www.bitlaw.com/copyright/database.html http://www.bitlaw.com/copyright/database.html http://www.bitlaw.com/source/cases/copyright/feist.html http://www.bitlaw.com/source/cases/copyright/feist.html If you read the second link, you'll see that it is exactly what the OP is doing and it has already been decided in favor of the database owner in case law.

colinsidoti 14y ago

I can't help but think you played this poorly. It sucks CL isn't going to let you get their data, but I really don't think this workaround is going to change anything (although kudos for the neat Google cache hack from 3taps). I imagine your development efforts would be better spent by leveraging the small but dedicated following you clearly have. There must be ways to take advantage of it without opening yourself up to more litigation.

ericd 14y ago
You may be right, I'm flying by the seat of my pants here (after having talked to some lawyers). My goal isn't to supplant Craigslist, which is what you seem to be suggesting - that's far too big a task to take on, even with my sites' substantial usage, and not really something I'm personally interested in - I think that classifieds are a mostly-solved problem, with the exception of reputation and security. I think CL can get there, they seem to be hiring devs.
- colinsidoti 14y ago
  Yea it sucks your dealing with CL and not Facebook or you probably would have been acquired. I say try and move the traffic to pad. It's certainly a big task, but not "far too big". Pick the region where you have the most significant traffic and start there. I haven't looked deeply into your site at all, but I know you have Garry Tans support, ask him what you need to do to get in YC. Get "Craigslist is the Killer" as a headline on TechCrunch. Piggyback the Padlister browser add-on with other common downloads, then push a popup to users posting apartment listings on craigslist saying "Also list on Padmapper?" There are all sorts of things you can try. There are hundreds of people on HN that wish they had the traffic you have. Use that momentum before it dies off. http://siteanalytics.compete.com/padmapper.com/ http://siteanalytics.compete.com/padmapper.com/
  - bluesnowmonkey 14y ago
    I think if you look up "network effect" in the dictionary, it's going to talk about Craigslist. Good luck overcoming that. A great strategy here would have been to establish a strong relationship with the people at Craigslist from the beginning. No idea if that was tried or not. Seriously, was the C&D letter the first communication exchanged between Padmapper and CL? What a missed opportunity. I wonder if this entire disaster could have been averted by calling the CL guys up early on and taking them out to lunch. Make friends, send them Christmas cards, then when this disagreement comes up you handle it over a round of golf instead of a C&D letter.

grabeh 14y ago

There's obviously a big fat indemnity clause in 3Taps' terms which means you're on the hook for claims made by third parties against 3Taps for loss arising as a result of the usage of the service. Having said that if CL provides access to 3Taps on terms which allow third parties to take data from 3Taps, then this may be permissible. It of course depends on the terms of 3Taps' access though (I admit to not being completely familiar with the ins and outs of 3Taps' service though). The problem of course is that CL have already shown a dislike of Padmapper's methods and as a result this workaround is unlikely to meet with CL's approval. Common sense would suffice to tell you that. Having said that, it's a real shame CL aren't open to discussions over licensing data (maybe they are of course but the price is too high).

ericd 14y ago
Nah, they weren't open to discussing any sort of deal involving the website (they were willing to license data for the mobile app).
- stcredzero 14y ago
  Why didn't you go that route? There are so many phones and tablets out there now, you'd have accomplished almost as much public good with far less controversy.
  - ericd 14y ago
    Good question. Basically, it's not really the best way for people to browse this kind of stuff, and there are still lots of people without phones/tablets. You can get a much better picture of things with PM maximized on a 20" screen than you can with a 4" one. The phone is more of a backup for when you're on the road or in a pinch.
    - stcredzero 14y ago
      The phone is more of a backup for when you're on the road or in a pinch. My girlfriend is looking for an apartment in Houston, and sometimes browses Craiglist with her iPad.

glesica 14y ago

I don't understand how CL can even exercise this kind of control. I'm not a lawyer, of course, but it seems reasonable to me that apartment listings would be considered "facts", just like baseball statistics. So then the CL terms of use are the only obstacle, but can you really be held to those if you didn't read them and weren't somehow forced/encouraged to do so? Perhaps so, but then I would find that a little scary. Just seems like a bad situation all around, but in my mind Padmapper has the moral (though perhaps not legal, again, not a lawyer) high ground, at least relatively speaking.

angryasian 14y ago
no its very simple. Don't submit your listing to craigslist them. Submit it to a site that is indexable friendly.
therandomguy 14y ago
Let me try. What you say is correct. The listing is "fact". And CL is not preventing posting this fact to other services. The user can post the same listing to CL, Rent.com and dozen other sites. What they are controlling is access to their servers.

Spooky23 14y ago

My question is: Why are people using craigslist? Fortunately for me, I purchased a home before all apartment ads migrated to CL. For the life of me, I don't get why it's so popular. The site is awful in at least a half dozen different ways.

dinedal 14y ago
It's free and easily accessible. Compare to the costs involved in getting an ad listed in a dead tree.
pavel_lishin 14y ago
> Why are people using craigslist? Because everyone else is.
sharkweek 14y ago
I feel like they snagged the community of online classifieds first and have been able to hold on pretty tightly, as people like my dad love that almost nothing has changed.
smackfu 14y ago
Ads migrated there because it was free, and a lot more agile than newspaper ads (wait a week for my ad to go live??). They stay because of network effects. And because it's good enough.
brittohalloran 14y ago
The funny thing is -- PadMapper and 3Taps and the like are actually perpetuating the usefulness of Craigslist: "everyone scrapes Craigslist, so if you just post it on Craigslist it will be pulled into all those other services automatically"
mrgoldenbrown 14y ago
Yes, the site is awful. But it is less awful than the alternatives. Therefore, people use it. Please, on behalf of all apartment hunters everywhere, (but especially in high tech/high rent places like the Bay area or Boston), tell us what the better option is! I say this in all seriousness. The idea that PadMapper was saving vast amount of man hours was not an exaggeration. I wasted many hours clicking on craigslist ads to find my current apartment.

ilamont 14y ago

The founder of 3Taps posted this on Quora last year, in response to a question about disrupting Craigslist: ... the postings in question are public facts about exchanges. Just as any price/supply/demand in a marketplace is open for any and all to notate and republish, so too is the entire set of Craigslist data -- as these offers between seekers and providers are clearly in the public domain. Historically, Craigslist has attempted to block access by others to the comprehensive use of this data. They block many 3rd parties who try to gain access to the data, and sometimes threaten to sue and bankrupt others as if they themselves created the underlying data and hold copyright like property rights over the same. But public facts are public property. And while some think that predatory Terms of Use demanding that you hand over the Brooklyn Bridge in liquidated damages if you don't comply with some obscure (and potentially constitutionally void) constraint will stand in court -- such absurdities will break if exposed. A fear, uncertainty,dread approach over access to data breaks down in a world where Google already indexes all of Craigslist data and caches that information all over the internet (for search performance results). If its possible and legal for Google, then why not for any and everyone else to also index and offer access to the same data. In short, Google doesn't get special secondary property rights to privatize public data to the exclusion of anyone else. Equal access to exchange data and search data is a principle in parallel to the notions of net neutrality. The points above are not a theoretical discourse. Look at 3taps.com/developers to see the execution of this concept. And look at what a 3rd party application (craiggers.com) can do in recreating the whole of Craigslist in a format that gives access to data in a way that is not remotely possible in the legacy Craigslist offering. Craiggers is a perfect example that the function of displaying Craigslist data (rather than gathering it) is a totally distinct (and competitive) marketplace, even if there are still huge network effects in the gathering of Craigslist postings. Note, Craiggers does NOT disrupt the existing Craigslist revenue model for Craigslist. It simply opens up the field (along with any other developer building on 3taps assisted access to Craigslist data) that wants to build on top of (rather than compete with) the network effects of Craigslist. Think Kayak and Indeed, but now for the whole body of data covered by Craigslist accessible, rather than just a single vertical. http://www.quora.com/Craigslist/Why-hasnt-another-product-disrupted-and-replaced-Craigslist/answer/Greg-Kidd http://www.quora.com/Craigslist/Why-hasnt-another-product-di...

14y ago

IanMechura 14y ago

I see some very interesting comments on this thread. I remember not long ago lots of people where upset that people where scraping stackoverflow and getting a higher ranking in google results. This was a threat to S/O's business model model and it made lots of people on this board uneasy. How is Pad mapper any different? How does the fact that this site has a better UI design somehow give them a pass to use data a company has spent millions of dollars gathering and feels is being used in a way that is a threat to their business model?

delackner 14y ago
Spent millions gathering? Two basic problems with that mindset. First, spending money to assemble a database of public facts does not magically give you a right to protect your investment in an uncopyrightable asset. Second, I don't know what millions you are talking about since craiglist's only asset is the users, and the users are just a result of tremendous network effects. They had a great idea and got very lucky, but gathering their massive userbase had nothing to do with spending money.
- delackner 14y ago
  Could someone explain what they disagree with? Do you disagree that databases of public facts are not copyrightable? With the idea that craigslist's investment in their system is irrelevant compared to the value of their userbase? With the idea that craigslist's main success is just being lucky?
base698 14y ago
Why would you even need to scrape it? It's available as a torrent: http://blog.stackoverflow.com/2009/06/stack-overflow-creative-commons-data-dump/ http://blog.stackoverflow.com/2009/06/stack-overflow-creativ...

stcredzero 14y ago

After reading the post and discussion, my conclusion is that Craigslist is making a policy mistake. They should make the same calculation of public good that ericd made and license 3rd party websites that add value to Craigslist, provided the 3rd parties do not harm the Craigslist infrastructure or business models. Doing otherwise is just holding up progress, and is a losing game in the long run. Facilitating value to users would be a win for them, as it was in the past.

debacle 14y ago
Creating a site where buyers can browse property listings does harm the Craigslist business model. To say otherwise expresses a lack of understanding of online ecosystems.
- stcredzero 14y ago
  As far as I can tell, they aren't posting the full listing. They are only aggregating the material facts about the listing. As far as I can tell, short of actually hoofing it to the place, you still have to get on Craigslist to make contact with the landlord.
  - hollerith 14y ago
    What Craigslist is afraid of (probably) is Padmapper becoming the first place most people turn to when looking for an apartment. Once that happens, Padmapper can cut Craigslist out of the loop by inviting landlords to make Padmapper listings instead of Craigslist listings. (Note that once Padmapper become to "go to" place for apartment hunters, Padmapper's refusing to continue to host or "index" Craigslist listings helps towards that goal.) Regardless the morality or legality of Padmapper's latest action, it is pretty clear to those of us who have spent a lot of time observing online ecosystems and software ecosystems that Padmapper's latest action is not in the interests of Craigslist since (even if Craigslist is not out to make money, but rather out to influence the world for the better) the only important source of Craigslist's potential for making money and of Craigslist's influence is Craigslist's being the first place people turn to when apartment hunting and doing other important things. That is why Craigslist is against Padmapper's latest action even if Padmapper never posts the full Craigslist listing.

dpcan 14y ago

I think everyone should just step back and look at the big picture. This guy is finding any means necessary to add Craiglist data back into his application and Craigslist does not want him to. I think the "gray" area is pretty black and white. Craigslist said no. He's doing it anyway - even if indirectly. Here's how I see it: Me: Don't go in my house. You: But you left the back door unlocked by accident, so I thought I'd just come in and eat your food. In the real world this would not be acceptable. Even though we are talking apples, bananas, and oranges, the same ethical perspective should be employed. Craig said no, so we respect his wishes. It's the RIGHT thing to do. If he adds an API someday, then he's sharing the key to his house with us, until then. Doors are locked, too bad, move on.

natrius 14y ago
It it invalid to apply analogies involving physical property to intellectual property. The two are not similar enough for that to be a convincing argument, especially since mere facts aren't protected intellectual property at all. http://en.wikipedia.org/wiki/Feist_v._Rural http://en.wikipedia.org/wiki/Feist_v._Rural
- dpcan 14y ago
  This seems to always be the reply when analogies come up, which is why I said it was apples, bananas, and oranges. The point being, what's mine is mine, and if I say you can't have it, I want you to respect my wishes. Sure you can probably justify endlessly, or interpret differently, or stretch the truth, or bend the rules, but one business is saying, don't use our data this way, and regardless of this, the other business is finding a way to use the data. Seems wrong to me. EDIT: Because I can't reply deeper to these threads, I have to edit here. The "mine" above is not intended to be in regards to the information or content on Craigslist. I'm not saying they own it. I think there is a relationship between the person who posted the information to Craigslist and Craigslist itself. I don't think anyone has a right to become some part of that relationship without both sides agreeing. Maybe it's because I'm a freelancer. If I have a relationship with a Client to build them something, and someone else decides, of their own accord, that they are now part of that relationship, I have the right to tell them to back off, and I believe they should respect my wishes. I'm not talking data ownership. I'm not talking about legal matters. I'm saying, human to human, they should just be cool with Craigslist wishes and go another way.

tubbo 14y ago

Dear PadMapper, We hate your app and you can't use our API. Sincerely, Craigslist + + + Dear Cragislist, Fuck you. Love, PadMapper

rdl 14y ago

Thank you for doing this. My only problem with it is that you're rewarding craigslist for being obnoxious tools, but ultimately it's better for the users, so that's an acceptable sacrifice. 3taps is pretty awesome. I wish they'd do an AMA or something.

raganwald 14y ago

Cartographers, encyclopedicians, and dictionary aggregators have always included some false data as a honeypot: An alleyway might not really exist. An obscure word might have been made up. A famous person might be entirely fictitious. The goal is to catch people copying their books. If you use “original research,” you’d never say that Braythwayt Street runs parallel to Rhodes Avenue in Toronto, you must have copied our map data without permission. I’m curious as to how padmapper intendeds to solve this problem. If Craigslist seed their listings with something false, that something is entirely craigslist’s copyright. It isn’t a “fact.” They don’t have a right to reproduce it. Lawyers, please chime in.

keithpeter 14y ago
Oh, short story idea there. Someone claims to live in the made up address and generates an identity complete with utility bills... Neal S here we come...
- raganwald 14y ago
  Fred Saberhagen used the concept of a map bunny as a plot device in one of his Berserker short stories.
mikeryan 14y ago
I believe padmapper is kind of relying on 3taps to take the hit. 3taps apparently is DMCA compliant. Long story short I'd expect 3taps to be craigslist legal target. (though actually I guess 3taps doesn't actually use the data so I'm not sure if what they're doing is technically illegal until padmapper uses it - I'm going to stop armchair lawyering now)
- deleted 14y ago
  [deleted]
- dpark 14y ago
  I'm not sure it matters what 3taps says or whether they are DMCA compliant. If I host pirated movies on my server and post in big bold letters, "I promise that you can redistribute them. It's totally legal and I'm responsible for any legal actions.", you're probably still getting sued by the MPAA if you redistribute the movies, and they'll probably win. I'm not saying Craigslist has any legitimate claim over the data, merely that if they do, accessing it via an indirect means probably doesn't change anything legally.

eli 14y ago

It's funny how many people are demanding that Craigslist sell out its user-generated content. I understand why people want this, but isn't CL being incredibly respectful of its users by allowing their data to be used only for the specific purpose for which it was posted and that's it?

crazygringo 14y ago
I don't think so. Users are on Craigslist to buy/sell, because it's the biggest place for that. If you post to CL, virtually everyone wants the largest audience possible, so 99.9% of people will be thrilled to have their post copied to other services. The only annoying thing would be if you take down your post from CL after you've bought/sold your thing, and you continue to get calls/emails from people getting it via copied data. But sites that copy just need to stay up-to-date on whether postings are still up or not...

seagreen 14y ago

What about a service that lets you post to every barter site at the same time, not just craigslist. Has that been tried? If you could get people to use it (through better UI and more exposure for what they're selling) you could break craigslist's hold on the marketplace.

duked 14y ago

Really, really stupid. I mean pretending you want to help people find apartment to justify you're action is ridiculous. You're running a business, be honest about it. Now craigislist do what they want with their data, you are just trying to build a service that "adds value" on top of someone else infrastructure( I mean "add" because I am one of the few who actually like the way cragislist display posts no bloaty js, no ugly pics no maps ... fast and simple).

JackC 14y ago
I find it difficult to believe that you have searched for an apartment in a tough rental market on Craigslist and consider their interface "fast and simple." If you've never had the task of finding an apartment within a mile of a given spot in a big metro area, you're not looking for what PadMapper offers. If you have had that problem, and you've never sworn out loud at Craigslist, you're a saint.
- duked 14y ago
  I was looking for an apartment 6 month ago in Reston, VA and I used craigslist for that purpose. I looked for a couple of zipcode (that I heard were good, since I was just moving in the US at that time) and honestly it worked great. Wouldn't it be the same in a big metro area ? just looking for interesting zipcodes ?
  - drstewart 14y ago
    You realize in some cities a few blocks makes a GIANT difference in neighborhood, right? Same zip code. Being able to visually filter those out is a major time saver.
    - JackC 14y ago
      The other issue is where I live, people put all kinds of random keywords in their Craigslist apartment ads to try to get attention. "Just 25 minutes' walk to Foo Square, Bar Square, and Baz Square! Four hours by horse to Lexington, and a short plane ride to London!" If I want to know what's within a 10-minute walk of a given location, I basically have to read every listing with my town's name in it every day and see if they included an address. Or spend 5 seconds glancing at PadMapper.

bambax 14y ago

How do 3taps do it? Some comments say they "use Google cache", which doesn't explain much: - Google is notorious for fighting bots pretty aggressively, how do 3taps avoid this? - even if 3taps found a special access to Google cache, it would be trivial for CL to prevent Google or anyone else from caching its listings, thus cutting off the rest of the food chain that feeds on caches; why don't they do it? Is it possible that 3taps use another method, such as "manually" copy-pasting listings with Mechanical Turk?

Zakuzaa 14y ago

http://traffic.alexa.com/graph?w=400&h=220&o=f&c=1&y=t&b=ffffff&n=666666&r=3m&u=padmapper.com&#38 http://traffic.alexa.com/graph?w=400&h=220&o=f&c...; Traffic took a great nosedive from the day they announced "bye bye" to CL.

huckleberries21 14y ago
By your own metric, so did traffic for CL, and CL is the 8th most visited site in the US, so their Alexa traffic data is probably more stable than traffic data for PadMapper. http://traffic.alexa.com/graph?w=400&h=220&o=f&c=1&y=r&b=ffffff&n=666666&r=3m&u=craigslist.org&#38 http://traffic.alexa.com/graph?w=400&h=220&o=f&c...;

rjsamson 14y ago

Not sure if this has been asked, but a philosophical question for Eric: if someone decided to start a padmapper competitor tomorrow, would you be okay with them using your data for free?

huckleberries21 14y ago
I don't see why not. By being a competitor that scrapes PM, they are accepting that there will be a delay in their data versus when it's available on PM. So unless their site was significantly better than PM in some way, in the really competitive apartment hunting markets, they would be strictly inferior to PM.

rprasad 14y ago

Consider, if the situation was reversed, and Craiglist was taking Padmapper's data without permission, would everyone on HN would be supporting Padmapper attempting to block Craiglist from reposting its listings? In such a scenario, Craiglist reposting the Padmapper listings would be a net gain for user's and posters (even though the posters never gave Craiglist such permission) because the listings would be exposed to an exponentially larger viewership. Being a startup is not a license (pun intended) to ignore other people's rights. Padmapper is in the wrong here, and if they proceed down this path they could face a ruinous (and fully justified) lawsuit.

lancefisher 14y ago

I appreciate that Eric is doing this in the open. The moral argument could go either way, but he is letting everyone know what he is doing and why. He deserves credit for inviting feedback on it. Personally, I hope padmapper does well.

jgh 14y ago

Padmapper is awesome, way way better than using Craiglist to apartment hunt, even if the data ultimately comes from Craigslist (among others). Maybe craigslist should buy padmapper instead of getting all lawsuity and take advantage of just how good their interface is for apartment hunting

pbreit 14y ago

For all the people mis-interpreting CL's TOUs: "You automatically grant and assign to CL, and you represent and warrant that you have the right to grant and assign to CL, a perpetual, irrevocable, unlimited, fully paid, fully sub-licensable (through multiple tiers), worldwide license to copy, perform, display, distribute, prepare derivative works from (including, without limitation, incorporating into other works) and otherwise use any content that you post. You also expressly grant and assign to CL all rights and causes of action to prohibit and enforce against any unauthorized copying, performance, display, distribution, use or exploitation of, or creation of derivative works from, any content that you post (including but not limited to any unauthorized downloading, extraction, harvesting, collection or aggregation of content that you post)."

etrain 14y ago

"Certainly by the time you read this." Code isn't live yet, at least for me, ericd - is this conversation giving you pause about deploying the new feature, or is it standard ops issues? Also, this couldn't come at a better time for me. I love padmapper and am moving to the bay area in 3 weeks, I need this, man!

ericd 14y ago
Sorry, was having some sysops issues, thanks for the heads up.

dredmorbius 14y ago

Eric's confusing method with goal. Craigslist's past method was to claim TOU violation on the part of PadMapper, denying PM ready use of its content. Its goal is to maintain market position as the dominant classifieds listing service. TOU is just one way for CL to enforce its market position, and as a commercial entity, it's very likely it will continue to do so. There's very little reason CL couldn't create an alternate legal claim based on content (the cartographic watermarking example comes to mind) to make multiple infringement claims against PM, which would invoke a whole host of copyright remedies ranging from injunctions and monetary damages to DMCA takedown notices (which could comprise the entire service depending on how construed by PM's hosting provider(s)). There's also the notion of a compilation copyright, in which a collection of CL postings could have standing. While Feist v. Rural Electric holds that a simple compilation of facts doesn't meet the minimal authorship requirements of copyright, a compilation of postings with some level of curation to CL might. I'm not aware of relevant caselaw here. My own suggestion would be that PM pursue a dual strategy of coming to a business agreement with CL (PM's interface is vastly superior) while curating its own set of rental listings independently (CL is ripe for disruption here). Disclaimer: I'm a long-time CL user and fan of much of its philosophy. I'm also a recent fan of PM. I hope the two will either work something out cooperatively, or manage to improve apartment and classifieds search by competitive means. I don't particularly care which.

Baba_Chaghaloo 14y ago

>I've always had a lot of respect for what they've done for the world Did I miss something? What has Craigslist done for the world besides put newspapers out of business?

genwin 14y ago
Using CL I furnished a whole house with super high-quality stuff for less than US$2K. Wouldn't have been realistic pre-CL since traditional classified ads don't have pictures. I also found my house through CL.

whyenot 14y ago

So, instead of directly scraping CL's data, they are now using an API that scrapes Google's cache of CL's data? That seems kind of sleazy.

trotsky 14y ago

I think it's perfectly fine if you want to play games dancing around what's legal and forge ahead against Craigslist's wishes. There's a long history of that kind of behavior in information services and they may or may not be able to stop you. But making a big deal out of being some kind of ux hero is pretty tacky. At the end of the day, no matter how small you are you're still a commercial enterprise gaining benefit at another commercial enterprise's expense. You're bidding on eyeballs with an offer of time savings. A vast majority of the web deliberately degrades their user experience in order to achieve business goals. Ads are almost always at least somewhat distracting, retailers stretch out checkout flows for up selling, sites "recommend" their high margin items ebay requires paypal and facebook sabotages their privacy management so you're less likely to bother. If CL was to build a bunch of sweet vertical specific apps for their sections they'd need a ton more developers, designers, testers, project managers and so on. To compensate for the greatly increased burn rate they'd need to monetize a lot better which almost always means features, content or policies that negatively impact the user. Or maybe they'd have just taken too much VC to pay for it and gone under in '02 when the bottom fell out. So by all means jockey away it's healthy for the industry. But crowing about it is a bit like taking a shit on the guy whose shoulders you're standing on while bragging about how tall you are.

ericd 14y ago
I'm not making a big deal about this, and I have a lot of respect for Craigslist, don't put words in my mouth. I think Craigslist is nearly perfect for the vast majority of things, I'm not on a crusade to "disrupt" them. It's just that for a long, drawn out, location-based search, it's objectively terrible, and I'm trying to patch that bug.
- J3L2404 14y ago
  http://news.ycombinator.com/item?id=4220106 http://news.ycombinator.com/item?id=4220106
- trotsky 14y ago
  It would seem like the definition of respect mostly conflicts with your behavior. re·spect 1. To feel or show deferential regard for; esteem. 2. to pay proper attention to; not violate "to respect Swiss neutrality" 3. to show consideration for; treat courteously or kindly

egillie 14y ago

Would a client-side padmapper be less morally gray?

declan 14y ago

I posted a look at the legal issues on Google+ here: https://plus.google.com/112961607570158342254/posts/HcCuHVG7rLM https://plus.google.com/112961607570158342254/posts/HcCuHVG7... Here's an excerpt from the post: Eric's blog post today says that "using 3taps just makes it so it’s not a TOU issue." The problem is it doesn't. Craigslist's lawyers already thought of this, which is why the TOU also prohibits the "display" and "derivative use" of "any content posted on craigslist," which PadMapper is clearly doing. As Craiglist's lawyer is likely to put it in the next C&D, assent to the TOU is still implied. [...] If 3taps is in fact scraping Google's cache, they're also violating Google's terms of use, which prohibits any effort "to access [search results] using a method other than the interface and the instructions that we provide." Anyone want to bet on how long it will be before Google blocks them through technical countermeasures, or sends its own C&D? And if 3taps is in the business of the TOU-prohibited "distribution" of Craigslist data, which it appears to be, it should expect to get a C&D from Craigslist too.