6 ms·

Is there any legal exposure possible? Like, a legitimate crawler suing you and alleging that you broke something of theirs?

by jawns 1y ago

bilekas 1y ago
Please, just as a conversational piece, walk me through the potentials you might think there are ? I'll play the side of the defender and you can play the "bot"/bot deployer.
- echoangle 1y ago
  Well creating a bot is not per se illegal, so assuming the maliciousness-detector on the server isn’t perfect, it could serve the zip bomb to a legitimate bot. And I don’t think it’s crazy that serving zip bombs with the stated intent to sabotage the client would be illegal. But I’m not a lawyer, of course.
  - bilekas 1y ago
    Disclosure, I'm not a lawyer either. This is all hypothetical high level discussion here. > it could serve the zip bomb to a legitimate bot. Can you define the difference between a legitimate bot, and a non legitimate bot for me ? The OP didn't mention it, but if we can assume they have SOME form of robots.txt (safe assumtion given their history), would those bots who ignored the robots be considered legitimate/non-legitimate ? Almost final question, and I know we're not lawyers here, but is there any precedent in case law or anywhere, which defines a 'bad bot' in the eyes of the law ? Final final question, as a bot, do you believe you have a right or a privilege to scrape a website ?
    - echoangle 1y ago
      > Can you define the difference between a legitimate bot, and a non legitimate bot for me ? Well by default every bot is legitimate, an illegitimate bot might be one that’s probing for security vulnerabilities (but I’m not even sure if that’s illegal if you don’t damage the server as a side effect, ie if you only try to determine the Wordpress or SSHD version running on the server for example). > The OP didn't mention it, but if we can assume they have SOME form of robots.txt (safe assumtion given their history), would those bots who ignored the robots be considered legitimate/non-legitimate ? robots.txt isn’t legally binding so I don’t think ignoring it makes a bot illegitimate. > Almost final question, and I know we're not lawyers here, but is there any precedent in case law or anywhere, which defines a 'bad bot' in the eyes of the law ? There might be but I don’t know any. > Final final question, as a bot, do you believe you have a right or a privilege to scrape a website ? Well I’m not a bot but I think I have the right to build bots to scrape websites (and not get served malicious content designed to sabotage my computer). You can decline service and just serve error pages of course if you don’t like my bot.
- brudgers 1y ago
  Anyone can sue anyone for anything and the side with the most money is most likely to prevail.
- pessimizer 1y ago
  Mantrapping is a fairly good analogy, and that's very illegal. If the person reading your gas meter gets caught in your mantrap, you're going to prison. You're probably going to prison if somebody burglarizing you gets caught in your mantrap. https://en.wikipedia.org/wiki/Mantrap_(snare) https://en.wikipedia.org/wiki/Mantrap_(snare) Of course their computers will live, but if you accidentally take down your own ISP or maybe some third-party service that you use for something, I'd think they would sue you.
bauruine 1y ago
>User-agent: * >Disallow: /zipbomb.html Legitimate crawlers would skip it this way only scum ignores robots.txt
- echoangle 1y ago
  I’m not sure that’s enough, robots.txt isn’t really legally binding so if the zip bomb somehow would be illegal, guarding it behind a robots.txt rule probably wouldn’t make it fine.
  - lcnPylGDnU4H9OF 1y ago
    Has any similar case been tried? I'd think that a judge learning the intent of robots.txt and disallow rules is fairly likely to be sympathetic. Seems like it could go either way, I mean. (Jury is probably more a crap-shoot.)
  - thephyber 1y ago
    Who, running a crawler which violates robots.txt, is going to prosecute/sue the server owner? The server owner can make an easy case to the jury that it is a booby trap to defend against trespassers.
    - dspillett 1y ago
      > can make an easy case to the jury that it is a booby trap to defend against trespassers I don't know of any online cases, but the law in many (most?) places certainly tends to look unfavourably on physical booby-traps. Even in the US states with full-on “stand your ground” legislation and the UK where common law allows for all “reasonable force” in self-defence, booby-traps are usually not considered self-defence or standing ground. Essentially if it can go off automatically rather than being actioned by a person in a defensive action, it isn't self-defence. > Who […] is going to prosecute/sue the server owner? Likely none of them. They might though take tit-for-tat action and pull that zipbomb repeatedly to eat your bandwidth, and they likely have more and much cheaper bandwidth than your little site. Best have some technical defences ready for that, as you aren't going to sue them either: they are probably running from a completely different legal jurisdiction and/or the attack will come from a botnet with little or no evidence trail wrt who kicked it off.
thayne 1y ago
Disclosure: IANAL The CFAA[1] prohibits: > knowingly causes the transmission of a program, information, code, or command, and as a result of such conduct, intentionally causes damage without authorization, to a protected computer; As far as I can tell (again, IANAL) there isn't an exception if you believe said computer is actively attempting to abuse your system[2]. I'm not sure if a zip bomb would constitute intentional damage, but it is at least close enough to the line that I wouldn't feel comfortable risking it. [1]: https://www.law.cornell.edu/uscode/text/18/1030 https://www.law.cornell.edu/uscode/text/18/1030 [2]: And of course, you might make a mistake and incorrectly serve this to legitimate traffic.
- jedberg 1y ago
  I don't believe the client counts as a protected computer because they initiated the connection. Also a protected computer is a very specific definition that involves banking and/or commerce and/or the government.
  - thayne 1y ago
    Part B of the definition of "protected computer" says: > which is used in or affecting interstate or foreign commerce or communication, including a computer located outside the United States that is used in a manner that affects interstate or foreign commerce or communication of the United States Assuming the server is running in the states, I think that would apply unless the client is in the same state as the server, in which case there is probably similar state law that comes into affect. I don't see anything there that excludes a client, and that makes sense, because otherwise it wouldn't prohibit having a site that tricks people into downloading malware.
    - jedberg 1y ago
      The word "accessed" is used multiple times throughout the law. A client accesses a server. A server does not access a client. It responds to a client. Also, the protected computer has to be involved in commerce. Unless they are accessing the website with the zip bomb using a computer that also is uses for interstate or foreign commerce, it won't qualify.
brudgers 1y ago
Though anyone can sue anyone, not doing X is the simplest thing that might avoid being sued for doing X. But if it matters pay your lawyer and if it doesn’t matter, it doesn’t matter.
klabb3 1y ago
Just crossed my mind that perhaps lots of bot traffic is coming from botnets of unaware victims who downloaded a shitty game or similar, orchestrated by a malicious C&C server somewhere else. (There was a post about this type of malware recently.) Now, if you crash the victims machine, it’s complicated at least ethically, if not legally.
- eru 1y ago
  Though ethically it might be a good thing to shut down their infected computer, instead of keeping it running.

Is there any legal exposure possible? Like, a legitimate crawler suing you and alleging that you broke something of theirs?

by jawns 1y ago

Is there any legal exposure possible? Like, a legitimate crawler suing you and alleging that you broke something of theirs?

bilekas 1y ago

Please, just as a conversational piece, walk me through the potentials you might think there are ? I'll play the side of the defender and you can play the "bot"/bot deployer.

echoangle 1y ago
Well creating a bot is not per se illegal, so assuming the maliciousness-detector on the server isn’t perfect, it could serve the zip bomb to a legitimate bot. And I don’t think it’s crazy that serving zip bombs with the stated intent to sabotage the client would be illegal. But I’m not a lawyer, of course.
- bilekas 1y ago
  Disclosure, I'm not a lawyer either. This is all hypothetical high level discussion here. > it could serve the zip bomb to a legitimate bot. Can you define the difference between a legitimate bot, and a non legitimate bot for me ? The OP didn't mention it, but if we can assume they have SOME form of robots.txt (safe assumtion given their history), would those bots who ignored the robots be considered legitimate/non-legitimate ? Almost final question, and I know we're not lawyers here, but is there any precedent in case law or anywhere, which defines a 'bad bot' in the eyes of the law ? Final final question, as a bot, do you believe you have a right or a privilege to scrape a website ?
  - echoangle 1y ago
    > Can you define the difference between a legitimate bot, and a non legitimate bot for me ? Well by default every bot is legitimate, an illegitimate bot might be one that’s probing for security vulnerabilities (but I’m not even sure if that’s illegal if you don’t damage the server as a side effect, ie if you only try to determine the Wordpress or SSHD version running on the server for example). > The OP didn't mention it, but if we can assume they have SOME form of robots.txt (safe assumtion given their history), would those bots who ignored the robots be considered legitimate/non-legitimate ? robots.txt isn’t legally binding so I don’t think ignoring it makes a bot illegitimate. > Almost final question, and I know we're not lawyers here, but is there any precedent in case law or anywhere, which defines a 'bad bot' in the eyes of the law ? There might be but I don’t know any. > Final final question, as a bot, do you believe you have a right or a privilege to scrape a website ? Well I’m not a bot but I think I have the right to build bots to scrape websites (and not get served malicious content designed to sabotage my computer). You can decline service and just serve error pages of course if you don’t like my bot.
brudgers 1y ago
Anyone can sue anyone for anything and the side with the most money is most likely to prevail.
pessimizer 1y ago
Mantrapping is a fairly good analogy, and that's very illegal. If the person reading your gas meter gets caught in your mantrap, you're going to prison. You're probably going to prison if somebody burglarizing you gets caught in your mantrap. https://en.wikipedia.org/wiki/Mantrap_(snare) https://en.wikipedia.org/wiki/Mantrap_(snare) Of course their computers will live, but if you accidentally take down your own ISP or maybe some third-party service that you use for something, I'd think they would sue you.

bauruine 1y ago

>User-agent: * >Disallow: /zipbomb.html Legitimate crawlers would skip it this way only scum ignores robots.txt

echoangle 1y ago
I’m not sure that’s enough, robots.txt isn’t really legally binding so if the zip bomb somehow would be illegal, guarding it behind a robots.txt rule probably wouldn’t make it fine.
- lcnPylGDnU4H9OF 1y ago
  Has any similar case been tried? I'd think that a judge learning the intent of robots.txt and disallow rules is fairly likely to be sympathetic. Seems like it could go either way, I mean. (Jury is probably more a crap-shoot.)
- thephyber 1y ago
  Who, running a crawler which violates robots.txt, is going to prosecute/sue the server owner? The server owner can make an easy case to the jury that it is a booby trap to defend against trespassers.
  - dspillett 1y ago
    > can make an easy case to the jury that it is a booby trap to defend against trespassers I don't know of any online cases, but the law in many (most?) places certainly tends to look unfavourably on physical booby-traps. Even in the US states with full-on “stand your ground” legislation and the UK where common law allows for all “reasonable force” in self-defence, booby-traps are usually not considered self-defence or standing ground. Essentially if it can go off automatically rather than being actioned by a person in a defensive action, it isn't self-defence. > Who […] is going to prosecute/sue the server owner? Likely none of them. They might though take tit-for-tat action and pull that zipbomb repeatedly to eat your bandwidth, and they likely have more and much cheaper bandwidth than your little site. Best have some technical defences ready for that, as you aren't going to sue them either: they are probably running from a completely different legal jurisdiction and/or the attack will come from a botnet with little or no evidence trail wrt who kicked it off.

thayne 1y ago

Disclosure: IANAL The CFAA[1] prohibits: > knowingly causes the transmission of a program, information, code, or command, and as a result of such conduct, intentionally causes damage without authorization, to a protected computer; As far as I can tell (again, IANAL) there isn't an exception if you believe said computer is actively attempting to abuse your system[2]. I'm not sure if a zip bomb would constitute intentional damage, but it is at least close enough to the line that I wouldn't feel comfortable risking it. [1]: https://www.law.cornell.edu/uscode/text/18/1030 https://www.law.cornell.edu/uscode/text/18/1030 [2]: And of course, you might make a mistake and incorrectly serve this to legitimate traffic.

jedberg 1y ago
I don't believe the client counts as a protected computer because they initiated the connection. Also a protected computer is a very specific definition that involves banking and/or commerce and/or the government.
- thayne 1y ago
  Part B of the definition of "protected computer" says: > which is used in or affecting interstate or foreign commerce or communication, including a computer located outside the United States that is used in a manner that affects interstate or foreign commerce or communication of the United States Assuming the server is running in the states, I think that would apply unless the client is in the same state as the server, in which case there is probably similar state law that comes into affect. I don't see anything there that excludes a client, and that makes sense, because otherwise it wouldn't prohibit having a site that tricks people into downloading malware.
  - jedberg 1y ago
    The word "accessed" is used multiple times throughout the law. A client accesses a server. A server does not access a client. It responds to a client. Also, the protected computer has to be involved in commerce. Unless they are accessing the website with the zip bomb using a computer that also is uses for interstate or foreign commerce, it won't qualify.

brudgers 1y ago

Though anyone can sue anyone, not doing X is the simplest thing that might avoid being sued for doing X. But if it matters pay your lawyer and if it doesn’t matter, it doesn’t matter.

klabb3 1y ago

Just crossed my mind that perhaps lots of bot traffic is coming from botnets of unaware victims who downloaded a shitty game or similar, orchestrated by a malicious C&C server somewhere else. (There was a post about this type of malware recently.) Now, if you crash the victims machine, it’s complicated at least ethically, if not legally.

eru 1y ago
Though ethically it might be a good thing to shut down their infected computer, instead of keeping it running.