3 ms·

Is scraping competitors’ public API’s out of the ordinary for technology companies? I was under the impression that Wal-Mart/Jet scraped Amazon and vice versa,

Is scraping competitors’ public API’s out of the ordinary for technology companies? I was under the impression that Wal-Mart/Jet scraped Amazon and vice versa, similar to how retail chains employ “secret shoppers” to discover competitors’ prices. I do remember a case where Craigslist sued a company for scraping its API and won. I wonder if it is invalidated by Weev’s subsequent acquittal. They could be apples/oranges since the former is civil while the latter is criminal. Also, Craigslist was able to demonstrate damages, which were clearly linked; every visitor to the other company’s page was one that didn’t visit Craigslist. And there are clear copyright issues with scraping Craigslist and using that user-generated information to populate another website.

alehul 9y ago
Though I'm unsure about competitors, there's a startup named ThinkNum which focuses entirely on scraping public APIs to provide investors with more data on tech companies.
malandrew 9y ago
Not only do competitor's do it, but also university and transportation authority researchers have as well. Usually scraping is covered under tort law as a trespass to chattels. Since services have limited capacity, any scraping that imposes an undue amount of traffic is essentially depriving that entity of using their capacity to serve legitimate customers. https://en.wikipedia.org/wiki/Trespass_to_chattels https://en.wikipedia.org/wiki/Trespass_to_chattels Insofar as copyright is concerned, you can't really copyright data, but you can copyright a collection of data. See Feist v. Rural. https://en.wikipedia.org/wiki/Feist_Publications,_Inc.,_v._Rural_Telephone_Service_Co https://en.wikipedia.org/wiki/Feist_Publications,_Inc.,_v._R.... To determine if someone is copying your data wholesale via scraping or some other method, it's not uncommon for a data provider to include false data that isn't real. If they see a competitor publishing that data, they can make the case that the competitor is copying their collection of data, especially if enough pieces of false data show up. In mapping, these are known as trap streets. https://en.wikipedia.org/wiki/Trap_street https://en.wikipedia.org/wiki/Trap_street
- linkregister 9y ago
  Ha! I love the idea of trap streets. From the wikipedia article, it appears that they're not protected by copyright in the U.S., though they are in the UK and Singapore.

alehul 9y ago

Though I'm unsure about competitors, there's a startup named ThinkNum which focuses entirely on scraping public APIs to provide investors with more data on tech companies.

malandrew 9y ago

Not only do competitor's do it, but also university and transportation authority researchers have as well. Usually scraping is covered under tort law as a trespass to chattels. Since services have limited capacity, any scraping that imposes an undue amount of traffic is essentially depriving that entity of using their capacity to serve legitimate customers. https://en.wikipedia.org/wiki/Trespass_to_chattels https://en.wikipedia.org/wiki/Trespass_to_chattels Insofar as copyright is concerned, you can't really copyright data, but you can copyright a collection of data. See Feist v. Rural. https://en.wikipedia.org/wiki/Feist_Publications,_Inc.,_v._Rural_Telephone_Service_Co https://en.wikipedia.org/wiki/Feist_Publications,_Inc.,_v._R.... To determine if someone is copying your data wholesale via scraping or some other method, it's not uncommon for a data provider to include false data that isn't real. If they see a competitor publishing that data, they can make the case that the competitor is copying their collection of data, especially if enough pieces of false data show up. In mapping, these are known as trap streets. https://en.wikipedia.org/wiki/Trap_street https://en.wikipedia.org/wiki/Trap_street

linkregister 9y ago
Ha! I love the idea of trap streets. From the wikipedia article, it appears that they're not protected by copyright in the U.S., though they are in the UK and Singapore.