3 ms·

> In any case, web scraping is a sort of gray area of the law. I don’t think it’s grey. It seems to be legal as the data are made freely available and the only

> In any case, web scraping is a sort of gray area of the law. I don’t think it’s grey. It seems to be legal as the data are made freely available and the only grey part is that companies don’t want this to happen and would rather charge and not have people scrape.

JohnFen 3y ago
It is a gray area (in the US) in the sense that there is no clear consensus about it in the courts. There have been court rulings in both directions.
- hallqv 3y ago
  What rulings are you referring to? Been reaffirmed that scraping LinkedIn is legal multiple times, even by Supreme Court. https://techcrunch.com/2022/04/18/web-scraping-legal-court/ https://techcrunch.com/2022/04/18/web-scraping-legal-court/
  - JohnFen 3y ago
    I believe that decision was that web scraping doesn't violate the CFAA unless there are access controls (such as the need to log in) to get to it, but they specifically said that the ruling doesn't comment about other possible claims against it. That's why I think it's still a gray area. But I could be wrong -- I haven't been following this stuff all that closely.
- SnowHill9902 3y ago
  So it’s not gray, it’s melange.
  - hallqv 3y ago
    Multiple court reaffirms in favor of scraping, including by the SC, seems more like verdant to me..
- prepend 3y ago
  What rulings have been against it? You don’t need consensus to allow something, you need consensus to be against something, otherwise it’s allowed. This “scraping is Grey” sounds like FUD to me. Legally, you can scrape anything publicly available as long as you damage the server. You understand that google is just a giant web scraper right? And all the other search engines spidering the web. Do you think they are risking their business being in the “grey?”
  - tyingq 3y ago
    There does seem to be some distinction if you log into and scrape: "Only three claims remained for the final order - the violation of the CAN-SPAM Act, violation of the CFAA and California Penal Code. The district court then granted summary judgment to Facebook on all three of the remaining Facebook claims. The district court awarded statutory damages of $3,031,350" https://en.wikipedia.org/wiki/Facebook,_Inc._v._Power_Ventures,_Inc https://en.wikipedia.org/wiki/Facebook,_Inc._v._Power_Ventur....
    - prepend 3y ago
      That’s completely different though as they are scraping private info that’s proprietary to Facebook. I think there’s no grey area for whether scraping publicly available data is legal as it is legal.
- blitzar 3y ago
  http get requests are legal.
ddingus 3y ago
It reminds me of a similar conflict in public records. Many municipalities charge for copies of public records. One can go to viewing rooms and examine the records at no charge. We all own those records as members of the public that funded them, and the place where they are kept. Many municipalities want to prohibit photography because people taking their own picture of a public record does not involve the copy fee. The municipality confuses access to records as a part of their fee, and that is where attempts to limit photography come from. The fee actually funds the work necessary for an official copy to be made, optionally stamped to be admissible in court or accepted as an "original copy." A photocopy of a death certificate is no different than the one the city clerk made, except for the stamp and clerk being able to testify about making the record copy. Some companies and government agencies will accept a death cert no matter what. Others want an official copy. Getting back to scraping: Clearly people have access and can make their own data copies. Maybe the answer is for companies to make official data products available, or something along those lines.

JohnFen 3y ago

It is a gray area (in the US) in the sense that there is no clear consensus about it in the courts. There have been court rulings in both directions.

hallqv 3y ago
What rulings are you referring to? Been reaffirmed that scraping LinkedIn is legal multiple times, even by Supreme Court. https://techcrunch.com/2022/04/18/web-scraping-legal-court/ https://techcrunch.com/2022/04/18/web-scraping-legal-court/
- JohnFen 3y ago
  I believe that decision was that web scraping doesn't violate the CFAA unless there are access controls (such as the need to log in) to get to it, but they specifically said that the ruling doesn't comment about other possible claims against it. That's why I think it's still a gray area. But I could be wrong -- I haven't been following this stuff all that closely.
SnowHill9902 3y ago
So it’s not gray, it’s melange.
- hallqv 3y ago
  Multiple court reaffirms in favor of scraping, including by the SC, seems more like verdant to me..
prepend 3y ago
What rulings have been against it? You don’t need consensus to allow something, you need consensus to be against something, otherwise it’s allowed. This “scraping is Grey” sounds like FUD to me. Legally, you can scrape anything publicly available as long as you damage the server. You understand that google is just a giant web scraper right? And all the other search engines spidering the web. Do you think they are risking their business being in the “grey?”
- tyingq 3y ago
  There does seem to be some distinction if you log into and scrape: "Only three claims remained for the final order - the violation of the CAN-SPAM Act, violation of the CFAA and California Penal Code. The district court then granted summary judgment to Facebook on all three of the remaining Facebook claims. The district court awarded statutory damages of $3,031,350" https://en.wikipedia.org/wiki/Facebook,_Inc._v._Power_Ventures,_Inc https://en.wikipedia.org/wiki/Facebook,_Inc._v._Power_Ventur....
  - prepend 3y ago
    That’s completely different though as they are scraping private info that’s proprietary to Facebook. I think there’s no grey area for whether scraping publicly available data is legal as it is legal.
blitzar 3y ago
http get requests are legal.

ddingus 3y ago

It reminds me of a similar conflict in public records. Many municipalities charge for copies of public records. One can go to viewing rooms and examine the records at no charge. We all own those records as members of the public that funded them, and the place where they are kept. Many municipalities want to prohibit photography because people taking their own picture of a public record does not involve the copy fee. The municipality confuses access to records as a part of their fee, and that is where attempts to limit photography come from. The fee actually funds the work necessary for an official copy to be made, optionally stamped to be admissible in court or accepted as an "original copy." A photocopy of a death certificate is no different than the one the city clerk made, except for the stamp and clerk being able to testify about making the record copy. Some companies and government agencies will accept a death cert no matter what. Others want an official copy. Getting back to scraping: Clearly people have access and can make their own data copies. Maybe the answer is for companies to make official data products available, or something along those lines.