15 ms·
Defending a website with Zip bombs
- colanderman 9y agoI would think Transfer-Encoding would be a better choice than Content-Encoding. It's processed at a lower level of the stack and must be decoded – Content-Encoding is generally only decoded if the client is specifically interested in whatever's inside the payload. (Note that you don't have to specify the large Content-Length in this case as it is implied by the transfer coding.) Also worth trying is an XML bomb [1], though that's higher up the stack. Of course you can combine all three in one payload (since it's more likely that lower levels of the stack implement streaming processing): gzip an XML bomb followed by a gigabyte of space characters, then gzip that followed by a gigabyte of NULs, then serve it up as application/xml with both Content-Encoding and Transfer-Encoding: gzip. (Actually now that I think of it, even though a terabyte of NULs compresses to 1 GiB [2], I bet that file is itself highly compressible, or could be made to be if it's handcrafted. You could probably serve that up easily with a few MiB file using the above technique.) EDIT: In fact a 100 GiB version of such a payload compresses down do ~160 KiB on the wire. (No, I won't be sharing it as I'm pretty sure that such reverse-hacking is legally not much different than serving up malware, especially since black-hat crawlers are more likely than not running on compromised devices.) [1] https://en.wikipedia.org/wiki/Billion_laughs https://en.wikipedia.org/wiki/Billion_laughs [2] https://superuser.com/questions/139253/what-is-the-maximum-compression-ratio-of-gzip/579290 https://superuser.com/questions/139253/what-is-the-maximum-c...
- amenghra 9y agoIf you are smart, you craft your compressed files to be infinite by making them Quines: https://alf.nu/ZipQuine https://alf.nu/ZipQuine
- cyounkins 9y agoIt's unlikely any of the parsers would apply recursive decompression, so a ZipQuine isn't going to help exhaust resources.
- amenghra 9y agoTrue. I wonder what's the largest file you can decompress to given some bound on the compressed result.
- pinpeliponni 9y agoAt least none of the Java libraries seemed to fall for this when I tried.
- vog 9y agoIndeed. ZipQuines are not targeted against servers, but against middleware. More precisely, these try to attack certain kinds of "security scanners". Those security scanners want to unpack as much as possible, by design(!), otherwise bad code could hide behind yet another round of compressed container formats.
- masklinn 9y agoYes scanners and AV is what's targetted by quines.
- masklinn 9y ago> EDIT: In fact a 100 GiB version of such a payload compresses down do ~160 KiB on the wire. (No, I won't be sharing it as I'm pretty sure that such reverse-hacking is legally not much different than serving up malware, especially since black-hat crawlers are more likely than not running on compromised devices.) http://www.aerasec.de/security/advisories/decompression-bomb-vulnerability.html http://www.aerasec.de/security/advisories/decompression-bomb... has a triple-gziped 100GB file down to 6k, the double-gzipped version is 230k. I'm trying on 1TB, but it turns out to take some time.
- Filligree 9y agoOne useful trick is that, for gzip, d(z(x+y)) = d(z(x) + z(y)). So you don't need to compress the entire terabyte.
- masklinn 9y agoI'd expect that to provide a lower compression, though it may not matter given the additional followup gzips. The compression finally finished after 3h (on an old MBP), "dd if=/dev/zero bs=1m count=1m | gzip | gzip | gzip" yields a bit under 10k (10082 bytes), and adding a 4th gzip yields a bit under 4k (4004 bytes). The 5th gzip starts increasing the size of the archive.
- Filligree 9y agoIt does, though I once used that trick to create a file containing more "Hello, World" lines than there are atoms in the universe. By, hmm, quite a large factor. It probably isn't a serious concern. It still fit on a floppy disk. :)
- colanderman 9y agoThat's true for the content stream but not gzip files themselves, which do have a minimal header.
- Filligree 9y ago
- jwilk 9y agoMost (all?) user-agents don't set the TE header (which specifies the transfer encodings the UA is willing to accept), so I doubt they would bother to decode unsolicited "Transfer-Encoding: gzip".
- colanderman 9y agoGood point, I didn't realize that.
- mnarayan01 9y agoThe Accept-Encoding header is used to specify the acceptable transfer encodings. Certainly all browsers set it, and most "advanced" frameworks will at least handle the common values of Transfer-Encoding regardless; a "malicious" crawler will almost certainly have to, as plenty of sites use Accept-Encoding along with User-Agent to block undesirable bots.
- colanderman 9y agoNo, Accept-Encoding pairs with Content-Encoding, not Transfer-Encoding (which is controlled by TE as GP indicated). [1] [2] [1] https://tools.ietf.org/html/rfc7231#section-5.3.4 https://tools.ietf.org/html/rfc7231#section-5.3.4 [2] https://tools.ietf.org/html/rfc7230#section-4.3 https://tools.ietf.org/html/rfc7230#section-4.3
- mnarayan01 9y agoBah I can't read...you two are correct.
- rixrax 9y agoI wonder how these techniques play along with random IDS/IPS/deep packet inspection/WAF/AV/DLP/web cache/proxy/load balancer/etc. devices that happen to look/peak into traffic that passes through network. I would wager my $ that more than a couple will need some admin(istrative) care after running some of this stuff via them. And btw -- when you end up accidentally crashing/DoS:ing your corporate WAF or ISPs DPI, who are they going to call?
- colanderman 9y agoI worked on an IPS a few years back. It was specifically designed NOT to inflate arbitrary ZIP files. All decoders worked in a streaming fashion and were limited to a fixed (low) nesting depth and preallocated memory. Typically they would be configured to reject traffic such as the payload we've been discussing.
- em3rgent0rdr 9y agoDefending by throwing things back at the attacker, instead of simply locking your door.
- leggomylibro 9y agoThis might actually work well with fail2ban integration. Every time you block a connection, you also respond to the final request with a big ol' file.
- ajarmst 9y agoThis is more that the thief is parked at your front door permanently trying to pick the lock, so you replace the valuables he's looking for with big chunks of lead.
- bulatb 9y agoMore like hungry bears.
- Pitarou 9y agoNo, he’s doing both.
- ianai 9y agoThis could also be seen as a bug on the browser side. I'd also be interested in the browser results for the petabyte version. I wonder if there's room to do this with other protocols? Ultimately we want to crash whatever tool the scriptkiddy uses.
- tyingq 9y agoI thought of http2's hpack. It does have built in protection though...the client sets a maximum header table size. Which encourages client implementations to think about it.
- avaer 9y agoDoes anyone know if this kind of white hat stuff has been tested by law? Because it seems in the realm of possibility that if a large botnet hits you and your responses crash a bunch of computers you could do serious time for trying it. I'm hoping there's precedent against this...
- ianai 9y agoA better outcome for an infected machine is complete failure than silent intrusion. The person then definitely knows something is wrong, AV software or not.
- microcolonel 9y agoI don't think there's a law against serving obscenely large files on the web, at least nowhere except Germany.
- komali2 9y ago>I don't think there's a law against... Connecting to a server...( A lot) Putting random strings into forms...( A lot) Moving your money between banks... (In different countries) Buying stocks... (With insider knowledge) A simple act doesn't spell the whole story, and fraud, computer crime, etc laws are written vaguely enough for a country to prosecute someone " sending large files."
- ajarmst 9y agoHe's got a pretty good defence in that all he's really doing is filtering requests and serving up a really large file to some of them. No active agency, and no executable code. If merely loading a large file crashes a computer, that's arguably the fault of the browser and/or OS.
- shakna 9y agoIntent really matters, especially in cases like these. He's serving up files deliberately, knowing they will likely cause problems. Microsoft doesn't take the fall for malware, even if its a fault in SMB or the like. The intent is damage.
- matt_wulfeck 9y agoThis is why web crawlers are built with upper boundaries on everything! Nobody malicious brings down crawlers. It's just unexpected things you find out on the internet.
- jacquesm 9y ago> Nobody malicious brings down crawlers. You're wrong about that. I've more than once brought down crawlers on purpose, especially the ones that didn't respect robots.txt.
- iopuy 9y agoWas there a reduction in ip's that Fail2ban would have picked up but instead were treated with the zip bomb?
- late2part 9y agobrilliant
- ajarmst 9y agoReminds me a bit of Upside-Down-Ternet: http://www.ex-parrot.com/pete/upside-down-ternet.html http://www.ex-parrot.com/pete/upside-down-ternet.html
- cypherpunks01 9y agoThis is like the soft equivalent of leaving a USBKill device in your backpack, to punish anyone who successfully steals it and tries to comb through your data.
- Pitarou 9y agoWay to make friends with the TSA. ;-)
- rocqua 9y agoIs this actually illegal? I'm sure they'd arrest you, but could they actually charge you? I guess if you see them try to use the USB killer, you'd be obligated to report it. Otherwise I don't think its an issue.
- danesparza 9y agoExcept that "I thought it would be fine" is a pretty reasonable answer at that point.
- Pitarou 9y agoRegardless of what the law says, in practice, breaking TSA equipment, interfering with TSA duties and, above all else, pissing off a TSA officer are all arrestable offences. If there’s any question about culpability, all they have to do is ask you, “Is there anything in your baggage you think we should know about?” and if you don’t disclose it then, you’re screwed. What? You say you were never asked such a question? You say you even tried to warn them? Well I have sworn testimony from a TSA officer that says they ALWAYS ask that question, and you’re the guy who was caught carrying a piece of equipment designed for trickery and vandalism. Case dismissed.
- oceanbreeze83 9y agodoesn't this incur large bandwidth data charges for the defender?
- zspitzer 9y agono, it's just sending a tiny zip file, decompression occurs at the other end
- chamakits 9y ago10 MB (the compressed GZIP given in the example) can be considerable. Even more so if you consider just how frequently bots are hitting those wp endpoints.
- toomuchtodo 9y agoNon-cloud providers don't rake you over the coals for transfer.
- DamnInteresting 9y agoThe author's 10MB example is far from optimized. From the article: > As 42.zip shows us it can compress a 4.5 peta byte (4.500.000 giga bytes) file down to 42 kilo bytes.
- banku_brougham 9y agoWe need some legal advice in this thread. What if the compressed file is plausibly valid content? How could intent be malicious if a request is served with actual content?
- eksemplar 9y agoIn this day and age, finding a vulnerability in a system like a mistakenly open API and running a script to call it a few times to investigate the weakness is considered hacking. It probably shouldn't be, but law is funny that way. Intentionally sending a zip bomb could potentially get you in trouble as well. Especially if you're just one private person or a small company without a legal division to brush it off. There isn't a real black/white interpretation though, at least not outside the US (where there may be history to influence ruling on the subject), and obviously most victims wouldn't report you, but more often than not you wouldn't want to test interpretation of IT related law.
- petre 9y agoAnother method is wasting attackers' time by sending out a character per second or so. It works so well for spam, that OpenBSD includes such a spamd honeypot.
- hossbeast 9y agoI would love to know how to configure this for ssh connection attempts
- yorwba 9y agoIf you are being adventurous, I guess you can just let them log in for a special user that has the shell set to a program that sends single characters very slowly. It is probably quite insecure, though.
- dredmorbius 9y agofail2ban, in a general sense.
- pmlnr 9y agoNo, this is a tarpit. Fail2ban simply rejects or drops vua iptables.
- dredmorbius 9y agohttps://gist.github.com/Belphemur/82d27b1b6dfd675d15f2 https://gist.github.com/Belphemur/82d27b1b6dfd675d15f2 Tarpit Action for Fail2ban with rate limit
- est 9y agoI have something similar on my VPS, edit /etc/issues.net to this Permission denied. Please try again.
- tyingq 9y agoSsh does support compression, but it seems to be only if the client requests it (ssh -C). You could, though, write a pam module to trickle data out very slowly. Maybe pam_python would be easier to experiment with. I use pam_shield to just null route ssh connections with X failed login attempts. There's no retaliation in that approach, but it does stop the brute forcing.
- eyuelt 9y agoThe article says that 42.zip compresses 4.5 petabytes down to 42 bytes. It should say 42 kilobytes. I don't see a way to comment on the article itself, but hopefully the author reads this.
- ben174 9y agoThank you. I was going crazy trying to think of what the contents of that 42 bytes would have been.
- geek_at 9y agothanks I fixed it
- philjohn 9y agoAnother small point, you don't need that starts with function, just use strpos === 0.
- jacquesm 9y agoIt's actually good practice to wrap a bit of code like that in a function with a name, as long as it isn't in some kind of extremely frequently executed inner loop. 'starts_with' is descriptive and language agnostic where '=== 0' is neither.
- hirsin 9y agoThis would be an entertaining way of dealing with MITM agents as well, over HTTP. As long as the client knows not to open the request, you could trade them back and forth with the MITM spy wasting tons of overhead.
- steego 9y agoIt would be an interesting way of streaming data if both sides used a custom decompression algorithm that skipped n bytes without allocating it anywhere. The payload could be encrypted text of two chat bots talking jibberish.
- mirimir 9y agoNow that's very interesting. Maybe hack a custom ssh with this feature. Adversaries that intercepted data or attempted MitM would be inconvenienced. Edit: Or even more useful, bbcp. Which is the best file transfer app that I've ever used.
- danesparza 9y agoThis is similar to kippo or cowrie, SSH honeypots: https://github.com/desaster/kippo https://github.com/desaster/kippo https://github.com/micheloosterhof/cowrie https://github.com/micheloosterhof/cowrie
- mirimir 9y agoI meant more than a honeypot. But rather, a functional SSH app that messes with standard SSH apps and libraries.
- shabble 9y agoSounds a bit like a simplified "Chaffing and Winnowing"[1], where the chaff identification is pre-shared through your custom compression parameters. There was a HN story[2] on Chaffinch[2], which is where I came across teh idea. [1] https://en.wikipedia.org/wiki/Chaffing_and_winnowing https://en.wikipedia.org/wiki/Chaffing_and_winnowing [2] https://news.ycombinator.com/item?id=14408757 https://news.ycombinator.com/item?id=14408757 [3] https://www.cl.cam.ac.uk/~rnc1/Chaffinch.html#Chaffing https://www.cl.cam.ac.uk/~rnc1/Chaffinch.html#Chaffing
- bberenberg 9y agoI think the natural next step is to make this into a Wordpress plugin.
- ruytlm 9y agoInteresting and related re attacks on a Tor hidden service: http://www.hackerfactor.com/blog/index.php?/archives/762-Attacked-Over-Tor.html http://www.hackerfactor.com/blog/index.php?/archives/762-Att... And the follow up: http://www.hackerfactor.com/blog/index.php?/archives/763-The-Continuing-Tor-Attack.html http://www.hackerfactor.com/blog/index.php?/archives/763-The...
- Laforet 9y agoVery entertaining read, thank you for sharing these. I used to dismiss black ICE [0] as cyberpunk tropes and it's good to be proven ignorant. [0]:https://en.wikipedia.org/wiki/Intrusion_Countermeasures_Electronics https://en.wikipedia.org/wiki/Intrusion_Countermeasures_Elec...
- vgb2k11 9y agoVery interesting read indeed. I've a question about it; the article is about defeating malicious crawlers/bots affecting a TOR hidden service, so my question is, how might the author differentiate bot requests from standard client requests on a request-by-request basis? I mean, can I assume that many kinds of requests arrive at hidden service through shared/common relays? Would this mean other fingerprinting methods (user agent etc) would be important, and if so, what options remain for the author if the attackers dynamically change/randomise their fingerprint on a per-request basis?
- compguy 9y agoWait a minute... He is doing the exact same thing as the former RaaS (ransomware as a service) operator Jeiphoos (he operated Encryptor RaaS). It's know that Jeiphoos is from Austria. Exactly one year after the shutdown of the service, someone from Austria is publishing the exactly same thing an Austrian ransomware operator were doing a year ago.
- jagermo 9y agoAha! the Hackernews detectives are on the case!
- deleted 9y ago[deleted]
- Scryptonite 9y agoReminds me of a time I once wrote a script in Node to send an endless stream of bytes at a slow & steady pace to bots that were scanning for vulnerable endpoints. It would cause them to hang, preventing them from continuing on to their next scanning job, some remaining connected for as long as weeks. I presume the ones that gave out sooner were manually stopped by whoever maintains them or they hit some sort of memory limit. Good times.
- reitanqild 9y agoI do the same to "Microsoft representatives" that call me because I have "lots of malware on my computer". Keep them on line by being a very dumb customer until they start cursing and hang up on me. : - )
- Gaelan 9y ago1-347-514-7296 is a phone number that automates this. Add it to a conference call, and frustrate the caller with no additional work. http://Reddit.com/r/itslenny http://Reddit.com/r/itslenny is the closest thing it has to an official site.
- throwanem 9y agoIt just rang forever when I tried it...
- Gaelan 9y agoAh, apparently you need to be whitelisted, my bad. There’s more information in the Reddit link. EDIT: Nope, apparently the person who runs it takes it down at night for some reason. Maybe to minimize people using it as a prank call?
- cdubzzz 9y agoYou do have to get whitelisted though, last I checked. I was able to use it one time and now if I call it I get a message about whitelisting. Same for JollyRoger.
- geek_at 9y agoHow come when I posted this (my blog post) here I only got 2 points? https://news.ycombinator.com/item?id=14704462 https://news.ycombinator.com/item?id=14704462 :D
- amenghra 9y agoMeritocracy /s
- peterburkimsher 9y agoYou can try writing "Show HN" at the start of your title, but that hasn't even helped me too much. I still get more upvotes/karma from writing witty comments instead of publishing code.
- em3rgent0rdr 9y agoa bit of a witty comment there...upvoted. :)
- Just1689 9y agoUpload time of day is also important
- vacri 9y agoAlso if it doesn't make it to the front page, it's hardly seen at all.
- dveeden2 9y agoDirectly serving /dev/zero or /dev/urandom also gives interesting results. (Be aware of bandwidth costs)
- vgb2k11 9y agoOh this seems quite an interesting experiment. Curious though if this defence poses no additional risks (beside bandwidth) on the server. I mean, is there any significant chance that the random data could cause a glitch on the server implementation?
- merricksb 9y agoSimilar topic a couple of months ago: https://news.ycombinator.com/item?id=14280084 https://news.ycombinator.com/item?id=14280084
- vacri 9y agoInteresting, on FF54 the test link pegs a CPU but the memory doesn't rise. Eventually it stops and CPU returns to normal. But then I did a 'view source', and the memory use rose until the browser got oomkilled (20GB free ram + swap)
- bluedino 9y agoI wonder is the browser is smart enough to not decompress when viewing or just uses it at a stream?
- aluhut 9y ago* Firefox: Memory rises up to 6-7gig, then just loads endlessly. Tab closable.
- brian-armstrong 9y agoDo browsers protect against media served with Content- or Transfer-Encoding like this? If you use something that lets you embed images, what's to stop you from crashing the browser of anyone who happens to visit the page your "image" is on?
- TazeTSchnitzel 9y agoNothing. I mean, crashing browsers with a client-side DoS is possible in many ways. With some horrible WebGL code I've crashed the macOS compositor before.
- c3833174 9y agoBrowsers alredy have massive codebases, I can't really imagine securing every non-security critical DoS vector
- Theizestooke 9y agoA great way... to provoke a war with people running botnets.
- deleted 9y ago[deleted]
- Pitarou 9y agoWow, you killed Tails! I tried visiting the payload site with Tails OS (a Linux distro for privacy minded) and the whole OS is frozen.
- TekMol 9y agoI don't think this "Defends" your website. If anything, it draws attention to it. Might also be used for some kind of reflection attack. Want to kill some service that let's users provide a url (for an avatar image or something) - point it to your zip bomber.
- oelmekki 9y agoTo be fair, people wanting to do that don't need author to have create a zip bomber, they can do it by themselves. Actually, I don't see how to defend this. Is there any way to ask a gzip file which size it will be once unzipped, without needing to decompress it?
- Qantourisc 9y agoYes, but your decompression middle ware might need an update/change: When you ask for a decompress, you specify the max size (if you are asking it to decompress everything).
- halomru 9y ago>Is there any way to ask a gzip file which size it will be once unzipped, without needing to decompress it? The closest is uncompressing it and counting and immediately discarding the bytes in the output stream. But of course the proper defense is to give up if you exceed a predefined memory or time budget.
- c3833174 9y agoDecompress it in chunks and stop when a preset limit is reached.
- danesparza 9y agoI think this is exactly what the HTTP 'HEAD' verb is for: https://developer.mozilla.org/en-US/docs/Web/HTTP/Methods/HEAD https://developer.mozilla.org/en-US/docs/Web/HTTP/Methods/HE...
- oelmekki 9y ago
- jacquesm 9y agoA friend of mine has a very useful little service that tracks attempts to breach servers from all over the world: https://www.blockedservers.com/ https://www.blockedservers.com/ It's a lot more effective to kill the connection rather than to start sending data if you're faced with a large number of attempts.
- leipert 9y agoInteresting page :) I'll write your friend an email with improvements for the site. The text shadow in the code blocks make them barely readable and the map color coding is bad for color blind people.
- jacquesm 9y agoI'm sure he will appreciate that. He's sysadmin at a Dutch IPSP and this is his side project.
- justusthane 9y agoIronically, it looks like the site has been DOSd by HN.
- futang44 9y agoJust tried it using piedpiper's middle-out algorithm and seeing astonishing results. It's so simple! D2F.1 = D2F.2, D2F.3 = D2F.4
- a1exus 9y agoyou better have unlimited bandwidth to try 10G))
- DamnInteresting 9y agoIn the example, the server only sends 10MB. The data is 10GB only after unzipping, which occurs on the client.
- DamonHD 9y agoInteresting! That also crossed with another thought about pre-compressing (real!) content so that Apache can serve it gzipped entirely statically with sendfile() rather than using mod_deflate on the fly, so unless I've misunderstood I think that bot defences can be served entirely statically to minimise CPU demand. I don't mind a non-checked-in gzip -v9 file of a few MB sitting there waiting... http://www.earth.org.uk/note-on-site-technicals.html http://www.earth.org.uk/note-on-site-technicals.html
- jscheel 9y agoBoth ZIP and GZIP file formats store the uncompressed filesize in their headers. You could stream and check for these headers to determine if the a zip bomb is being delivered. Obviously something script-kiddies aren't going to do, but the scripts they use can be improved and redistributed fairly easily.
- vgb2k11 9y agoCould the head be spoofed in such a way that the header says 1MB, or might the clients/bots be typically strict on ensuring header values are valid? I think your raised issue is important though, and any serious client/bot should be ignoring files with 1KB -> 1GB decompression ratios.
- wooptoo 9y agoA similar `slow bomb` could be created for attempted ssh connections to a host using a sshrc script. For example clients which do not present a key, just keep them connected and feed them garbage from time to time. Or rickroll them.
- ilurkedhere 9y agoWouldn't all but the most naive scanners use time-out settings, maximum lengths on bytes read etc?
- vgb2k11 9y ago> Wouldn't all but the most naive scanners use time-out settings, maximum lengths on bytes read etc? It wouldn't save a scanner from crashing to use a time-out or max read bytes. The defense can send the 100kb zipped data in a matter of seconds. The client then decompresses the zipped data which expands to gigabytes, causing crashes by out-of-memory.
- ilurkedhere 9y agoWas thinking more about a maximum length for the decompression stage.
- vgb2k11 9y agoUser ruytlm has posted links to hacker factor blog, and it seems some sophisticated scanners (e.g., Eddie) were crashed by the exploit. In that blog the author postulates that Eddie is a nation-state level (not script kiddie) scanner, so I'd say that the answer to this question will be in your definition of naive. It's tempting to qualify any scanner which crashes on this as naive though, I'd agree. Especially moving forward with the publicity of this post/topic. Well actually from memory the author of the blog was doubtful if this exploit actually crashed Eddie or not, but it did crash the other bots (Eddie V1 did go offline, possibly as a crash), so it would appear you are correct. Only truely naive bots might well be affected by this.
- fmavituna 9y agoWe are developing a web application security scanner [1] and we indeed use max length setting and also detect binary responses, just tested this and as expected it worked fine. I'm actually surprised that many other scanners failed to do this. [1] https://www.netsparker.com https://www.netsparker.com
- ioquatix 9y agoAbout a month ago one of my websites was being scraped. They were grabbing JSON data from a mapping system. I replaced it with a GZIP bomb. It was very satisfying to watch the requests start slowing down, and eventually stop.
- glenscott1 9y agoWhat are good strategies for protecting your website against ZIP bomb file uploads?