4 ms·

> what they are doing is illegal It's not illegal. Google can sue them and bury them in court fees and potentially win a civil suit, but it sure as hell isn't

by meritt 6y ago

> what they are doing is illegal It's not illegal. Google can sue them and bury them in court fees and potentially win a civil suit, but it sure as hell isn't illegal.

fxtentacle 6y ago
I'm pretty sure this is legal wrt CFAA because they are not circumventing any access control mechanism. However, 300k requests per day surely is enough that this could be considered a sort of denial of service attack and violate fair use. If Google wanted to, they would scale down their servers for a day, wait for this traffic peak to hit, document how it made the service unavailable to others, and now you have a valid offense to sue for.
- meritt 6y ago
  > now you have a valid offense to sue for. Yes. That's also what I said above when I said "Google can sue them". It's not illegal is my entire point. You can sue anyone, for any reason, at any time, and even win. That doesn't make something illegal. If Google sends them a C&D, and expressly forbids them from doing this activity, and implements technical measures to prevent them from doing so, and they continue doing so then they may start approaching the area of illegal (Craigslist v. 3Taps would agree, hiQ v. LinkedIn would disagree).
- bdcravens 6y ago
  > what they are doing is illegal illegal means violate criminal code. > now you have a valid offense to sue for That's civil, not criminal. It's pedantic, but words matter.
- occamrazor 6y ago
  They are using residential proxies to evade rate-limiting by google. I don’t know if it’s enough to trigger CFAA, but it shows at a minimum that they know that what they are doing is not what google considers fair use.
- tmerr 6y ago
  I am surprised people think this could affect the servers? If they scrape 300k prices from 25k pages a day, and the crawl runs every hour from 15-22 UTC, then that means the 25k is spread across 3.5k pages per crawl. Even if the crawl is aggressive and completes within a minute that is still only 58 QPS.
- fragmede 6y ago
  The court decision in the hiQ vs LinkedIn case limits the CFAA's reach and scraping of publicly accessible data "likely does not violate the CFAA". https://www.eff.org/deeplinks/2019/09/victory-ruling-hiq-v-linkedin-protects-scraping-public-data https://www.eff.org/deeplinks/2019/09/victory-ruling-hiq-v-l...
- sdinsn 6y ago
  > denial of service attack 300k requests / day is a little over 3 per second. That's not much.

by meritt 6y ago

> what they are doing is illegal It's not illegal. Google can sue them and bury them in court fees and potentially win a civil suit, but it sure as hell isn't illegal.

fxtentacle 6y ago

I'm pretty sure this is legal wrt CFAA because they are not circumventing any access control mechanism. However, 300k requests per day surely is enough that this could be considered a sort of denial of service attack and violate fair use. If Google wanted to, they would scale down their servers for a day, wait for this traffic peak to hit, document how it made the service unavailable to others, and now you have a valid offense to sue for.

meritt 6y ago
> now you have a valid offense to sue for. Yes. That's also what I said above when I said "Google can sue them". It's not illegal is my entire point. You can sue anyone, for any reason, at any time, and even win. That doesn't make something illegal. If Google sends them a C&D, and expressly forbids them from doing this activity, and implements technical measures to prevent them from doing so, and they continue doing so then they may start approaching the area of illegal (Craigslist v. 3Taps would agree, hiQ v. LinkedIn would disagree).
bdcravens 6y ago
> what they are doing is illegal illegal means violate criminal code. > now you have a valid offense to sue for That's civil, not criminal. It's pedantic, but words matter.
occamrazor 6y ago
They are using residential proxies to evade rate-limiting by google. I don’t know if it’s enough to trigger CFAA, but it shows at a minimum that they know that what they are doing is not what google considers fair use.
tmerr 6y ago
I am surprised people think this could affect the servers? If they scrape 300k prices from 25k pages a day, and the crawl runs every hour from 15-22 UTC, then that means the 25k is spread across 3.5k pages per crawl. Even if the crawl is aggressive and completes within a minute that is still only 58 QPS.
fragmede 6y ago
The court decision in the hiQ vs LinkedIn case limits the CFAA's reach and scraping of publicly accessible data "likely does not violate the CFAA". https://www.eff.org/deeplinks/2019/09/victory-ruling-hiq-v-linkedin-protects-scraping-public-data https://www.eff.org/deeplinks/2019/09/victory-ruling-hiq-v-l...
sdinsn 6y ago
> denial of service attack 300k requests / day is a little over 3 per second. That's not much.