3 ms·
Marketplace data would be best if it were fully decentralized. At a company I am doing software consulting for we've been building bits and pieces of a distrib
by malandrew 14y ago
Marketplace data would be best if it were fully decentralized.
At a company I am doing software consulting for we've been building bits and pieces of a distributed scraper and one of the ideas we see as a possible approach to solving these data silos is the equivalent of generalized "bit torrent for API data". So long as your service does not publish the copyrightable content gathered via API data or at least does so in a way that respects the DMCA requirements, you are pretty much free any clear, since a bittorrent approach would completely absolve you from agreeing to any terms of use and the distributed approach would absolve you from a trespass to chattels lawsuit. Between those two and the Feist vs. Rural ruling, I cannot think of any defense a company could mount.
Near as I can tell both Twitter and Craigslist (among many many others) are all vulnerable to this data collection vector.
If this problem sounds interesting to you, shoot me an email (in my profile). I know they are hiring.
- paulgb 14y agoI think CL abandoned the trespass to chattels argument when PadMapper switched to 3taps. Couldn't CL file DMCA requests given the exclusivity agreement they now have with their users? I agree that marketplace data should be decentralized. I hope it works out for you.
- dangrossman 14y agoFacts aren't protected by copyright, and PadMapper only distributes facts (# of bedrooms, location and price). It doesn't copy the CL post title, description or photos which would be protected. I don't think there's any copyright infringement to serve a DMCA request against, but I'm no lawyer.
- pbreit 14y agoTwitter and Craigslist (among many many others) are all vulnerable Do you realize how ridiculous that sounds?
- malandrew 14y agoWhy?
- pbreit 14y agoThose are two of the very least vulnerable web properties. And all the evidence points to centralization being the cornerstone to their existence and outrageous success. In one breath you've put forth a relatively weak idea and made an egregiously bad evaluation of some incumbents.
- i_s 14y agoWhat he is talking about is the potential of getting these sites' data to third parties without them having to ask for permission. What evidence suggests that this idea is weak?
- malandrew 14y agoLike i_s said, we're only worried about getting data out, not writing data to. Craigslist and Twitter both have read and write monopolies of in the two sided market of classified ads and interpersonal 140-character messages, respectively. We're not trying to do anything to displace them from their write monopoly. We're coming up with a technological solution that will hopefully be like taking a cat out of a bag. Once you take it out, hopefully you can't put the cat back in. However, with enough self-interested nodes forming a swarm to extract and share the data, essentially creating a publicly accessible ToS free copy of the data, you have basically broken the read monopoly those and other web properties have. From there, it's much easier for many agents to form a federated network of writes to compete with the read monopoly.