4 ms·

I’m not sure that’s enough, robots.txt isn’t really legally binding so if the zip bomb somehow would be illegal, guarding it behind a robots.txt rule probably w

I’m not sure that’s enough, robots.txt isn’t really legally binding so if the zip bomb somehow would be illegal, guarding it behind a robots.txt rule probably wouldn’t make it fine.

lcnPylGDnU4H9OF 1y ago
Has any similar case been tried? I'd think that a judge learning the intent of robots.txt and disallow rules is fairly likely to be sympathetic. Seems like it could go either way, I mean. (Jury is probably more a crap-shoot.)
thephyber 1y ago
Who, running a crawler which violates robots.txt, is going to prosecute/sue the server owner? The server owner can make an easy case to the jury that it is a booby trap to defend against trespassers.
- dspillett 1y ago
  > can make an easy case to the jury that it is a booby trap to defend against trespassers I don't know of any online cases, but the law in many (most?) places certainly tends to look unfavourably on physical booby-traps. Even in the US states with full-on “stand your ground” legislation and the UK where common law allows for all “reasonable force” in self-defence, booby-traps are usually not considered self-defence or standing ground. Essentially if it can go off automatically rather than being actioned by a person in a defensive action, it isn't self-defence. > Who […] is going to prosecute/sue the server owner? Likely none of them. They might though take tit-for-tat action and pull that zipbomb repeatedly to eat your bandwidth, and they likely have more and much cheaper bandwidth than your little site. Best have some technical defences ready for that, as you aren't going to sue them either: they are probably running from a completely different legal jurisdiction and/or the attack will come from a botnet with little or no evidence trail wrt who kicked it off.
  - thephyber 1y ago
    The illegality of boobytrapping your house appears to be illegal because of the potential threat to life/health. A zip bomb doesn’t threaten any people. At worst, it can fill up memory and storage on a device. I’m pretty sure it wouldn’t violate any of the same statutes and it most likely wouldn’t fall nicely under any of the common law jurisprudence that you mentioned. > pull that zipbomb repeatedly to eat your bandwidth, and they likely have more and much cheaper bandwidth than your little site. Go read what a zip bomb is. There is one that is only a few KB, which is comparable in server load + bandwidth to a robots.txt.
    - dspillett 1y ago
      > Go read what a zip bomb is. No need to be a dick. Especially when you yourself are in the process of not understanding what others are saying. I know full well what a zipbomb is. A large compressed file still has some size even in compressed form (without nesting, 1G of minimal entropy data is ~1M gzipped). If someone has noticed your bomb and worked around it by implementing relevant checks (or isn't really affected by it because of already having had those checks in place), they can get a little revenge by soaking up your bandwidth downloading it many times. OK, so nested that comes down to a few Kb, they can still throw a botnet at that, or some other content on your site, and cause you some faf, if they wish to engage in tit-for-tat action. Also: nesting doesn't work when you are using HTTP transport compression as your delivery mechanism, which is what is being discussed here: “standard” libraries supporting compressed HTTP encodings don't generally unpack nested content. There is no “Accept-Encoding: gzip+gzip” or similar. Most, perhaps the vast majority, won't care to make the effort, so this could be considered a hypothetical, but some might. There were certainly cases, way back in my earlier days online, of junk mailers and address scrapers deliberately wasting bandwidth of sites that encouraged the use of tools like FormFucker or implemented scraper sinkholes.
boricj 1y ago
> robots.txt isn’t really legally binding Neither is the HTTP specification. Nothing is stopping you from running a Gopher server on TCP port 80, should you get into trouble if it happens to crash a particular crawler? Making a HTTP request on a random server is like uttering a sentence to a random person in a city: some can be helpful, some may tell you to piss off and some might shank you. If you don't like the latter, then maybe don't go around screaming nonsense loudly to strangers in an unmarked area.
- echoangle 1y ago
  The law might stop you from sending specific responses if the only goal is to sabotage the requesting computer. I’m not 100% familiar with US law but I think intentionally sabotaging a computer system would be illegal.
  - seqizz 1y ago
    I'm also not a lawyer, but wouldn't they dismiss this as a sabotage if the requester is not legally forced to request it in the first place?
    - echoangle 1y ago
      No, why would they? If I voluntarily request your website, you can’t just reply with a virus that wipes my harddrive. Even though I had the option to not send the request. I didn’t know that you were going to sabotage me before I made the request.
      - seqizz 1y ago
        Because you requested it? There is no agreement on what or how to serve things, other than standards (your browser expects a valid document on the other side etc). I just assumed court might say there is a difference between you requesting all guess-able endpoints and find 1 endpoint which will harm your computer (while there was _zero_ reason for you to access that page) and someone putting zipbomb into index.html to intentionally harm everyone.
eru 1y ago
The law generally rewards good faith attempts, and robots.txt is an established commercial standard.

I’m not sure that’s enough, robots.txt isn’t really legally binding so if the zip bomb somehow would be illegal, guarding it behind a robots.txt rule probably wouldn’t make it fine.

lcnPylGDnU4H9OF 1y ago

Has any similar case been tried? I'd think that a judge learning the intent of robots.txt and disallow rules is fairly likely to be sympathetic. Seems like it could go either way, I mean. (Jury is probably more a crap-shoot.)

thephyber 1y ago

Who, running a crawler which violates robots.txt, is going to prosecute/sue the server owner? The server owner can make an easy case to the jury that it is a booby trap to defend against trespassers.

dspillett 1y ago
> can make an easy case to the jury that it is a booby trap to defend against trespassers I don't know of any online cases, but the law in many (most?) places certainly tends to look unfavourably on physical booby-traps. Even in the US states with full-on “stand your ground” legislation and the UK where common law allows for all “reasonable force” in self-defence, booby-traps are usually not considered self-defence or standing ground. Essentially if it can go off automatically rather than being actioned by a person in a defensive action, it isn't self-defence. > Who […] is going to prosecute/sue the server owner? Likely none of them. They might though take tit-for-tat action and pull that zipbomb repeatedly to eat your bandwidth, and they likely have more and much cheaper bandwidth than your little site. Best have some technical defences ready for that, as you aren't going to sue them either: they are probably running from a completely different legal jurisdiction and/or the attack will come from a botnet with little or no evidence trail wrt who kicked it off.
- thephyber 1y ago
  The illegality of boobytrapping your house appears to be illegal because of the potential threat to life/health. A zip bomb doesn’t threaten any people. At worst, it can fill up memory and storage on a device. I’m pretty sure it wouldn’t violate any of the same statutes and it most likely wouldn’t fall nicely under any of the common law jurisprudence that you mentioned. > pull that zipbomb repeatedly to eat your bandwidth, and they likely have more and much cheaper bandwidth than your little site. Go read what a zip bomb is. There is one that is only a few KB, which is comparable in server load + bandwidth to a robots.txt.
  - dspillett 1y ago
    > Go read what a zip bomb is. No need to be a dick. Especially when you yourself are in the process of not understanding what others are saying. I know full well what a zipbomb is. A large compressed file still has some size even in compressed form (without nesting, 1G of minimal entropy data is ~1M gzipped). If someone has noticed your bomb and worked around it by implementing relevant checks (or isn't really affected by it because of already having had those checks in place), they can get a little revenge by soaking up your bandwidth downloading it many times. OK, so nested that comes down to a few Kb, they can still throw a botnet at that, or some other content on your site, and cause you some faf, if they wish to engage in tit-for-tat action. Also: nesting doesn't work when you are using HTTP transport compression as your delivery mechanism, which is what is being discussed here: “standard” libraries supporting compressed HTTP encodings don't generally unpack nested content. There is no “Accept-Encoding: gzip+gzip” or similar. Most, perhaps the vast majority, won't care to make the effort, so this could be considered a hypothetical, but some might. There were certainly cases, way back in my earlier days online, of junk mailers and address scrapers deliberately wasting bandwidth of sites that encouraged the use of tools like FormFucker or implemented scraper sinkholes.

boricj 1y ago

> robots.txt isn’t really legally binding Neither is the HTTP specification. Nothing is stopping you from running a Gopher server on TCP port 80, should you get into trouble if it happens to crash a particular crawler? Making a HTTP request on a random server is like uttering a sentence to a random person in a city: some can be helpful, some may tell you to piss off and some might shank you. If you don't like the latter, then maybe don't go around screaming nonsense loudly to strangers in an unmarked area.

echoangle 1y ago
The law might stop you from sending specific responses if the only goal is to sabotage the requesting computer. I’m not 100% familiar with US law but I think intentionally sabotaging a computer system would be illegal.
- seqizz 1y ago
  I'm also not a lawyer, but wouldn't they dismiss this as a sabotage if the requester is not legally forced to request it in the first place?
  - echoangle 1y ago
    No, why would they? If I voluntarily request your website, you can’t just reply with a virus that wipes my harddrive. Even though I had the option to not send the request. I didn’t know that you were going to sabotage me before I made the request.
    - seqizz 1y ago
      Because you requested it? There is no agreement on what or how to serve things, other than standards (your browser expects a valid document on the other side etc). I just assumed court might say there is a difference between you requesting all guess-able endpoints and find 1 endpoint which will harm your computer (while there was _zero_ reason for you to access that page) and someone putting zipbomb into index.html to intentionally harm everyone.

eru 1y ago

The law generally rewards good faith attempts, and robots.txt is an established commercial standard.