15 ms·
15 years later: remote code execution in qmail
- pfortuny 6y agoOh, dear. Another "will never happen" issue which, after not so long (in the history of mankind) takes place. And this one would not have been so complicated to fix (just enforce a limit as the author suggests). /* this line is unreachable */ printf("You have reached unreachable code\n"); 10 years later, it gets printed...
- bluejekyll 6y agoWhy not exit/panic at that point? Seems like the right thing for any “unreachable” code.
- Eridrus 6y agoEgo.
- kevincox 6y agoIf you are really sure put __builtin_unreachable()
- pvarangot 6y agoI'm pretty sure he didn't have that on most of the platforms he was targeting with his portable C code, it's not even there on some of them now.
- jstimpfle 6y agowhat you do is you define a macro in a common header. Like UNREACHABLE().
- jstimpfle 6y agoShoot I have done that in the past but it seems I actually wanted __builtin_trap().
- kevincox 6y agoYeah, that footgun should really have a more explicit name. __dear_compiler_i_promise_that_this_statement_is_unreachable_and_you_may_optimize_based_on_my_good_word_upon_penalty_of_undefined_behaviour()
- derefr 6y agoI mean, what other semantics would anyone expect it to have—presuming they think for a moment about why it exists as its own thing in the first place?
- dodobirdlord 6y agoRust being Rust, the equivalent causes a safe teardown. https://doc.rust-lang.org/std/macro.unreachable.html https://doc.rust-lang.org/std/macro.unreachable.html The UB causing equivalent is of course unsafe. https://doc.rust-lang.org/std/hint/fn.unreachable_unchecked.html https://doc.rust-lang.org/std/hint/fn.unreachable_unchecked....
- jstimpfle 6y agoOk so what's the answer if you apply the question to __builtin_trap()?
- saagarjha 6y agoEven better, just actually perform undefined behavior on that branch and the compiler will get rid of it.
- kevincox 6y agoThis is literally what __builtin_unreachable() does. But it has the added benefit of the compiler understanding the intent so it works well with diagnostics.
- badrabbit 6y agoBecause it will never happen I guess. From my experience, it's because of this bad habbit of expecting the program to crash and provide a backtrace,the print is there for debugging. But you're right,it's not the sane way of doing it.
- im3w1l 6y agoDidn't think of it? If I saw a line like that I'd be grateful that it printed at all.
- asveikau 6y agoI could see an argument (similar to for assert()) to omit such a check in most production builds. Maybe you could enable it for a small amount of hosts, so that if it does happen with any frequency, somebody sees it somewhere. But not every "unthinkable" case being reached represents a security problem as it does in this case. So in the general case of how to approach such assert-type checks, an abort might take down the process when not necessary, turning it into a DoS. Edit: I am not sure if my friend the downvoter realizes I am not trying to justify legit security bugs, just talking abstractly about varying approaches to handling asserts for unforeseen or "allegedly impossible" circumstances.
- kstrauser 6y agoI didn't downvote you and I interpreted your comment the way you said you meant it. I do happen to disagree, though. For a project in a space that's notoriously vulnerability prone - I mean, when it was released it was competing with Sendmail - it seems very reasonably cautious to pepper the code with panic handlers in "impossible" places. They won't slow the code down any because they should never be evaluated, but give nice, noisy explosions when unexpected things happen. There's no downside to doing this.
- chasil 6y agoSolaris also notably bundles 32-bit binaries for most everything in /bin with 64-bit distributions, and that begins to look like a wise choice. I recently observed that SmartOS does the same.
- tedunangst 6y agoIf we could tell a priori which unreachable cases were reachable it wouldn't be such a problem. Asserting only in debug builds is rather ineffective because the unthinkable input isn't in your testsuite. If it were, then it wouldn't have been unthinkable.
- asveikau 6y ago> Asserting only in debug builds Right, but I am also suggesting another approach: if you have a large fleet of machines or a client app with a lot of usage, you can enable it in a small percentage of runs and if it hits in the wild you still have a chance of learning about it, without tearing down the process for all users. I have seen that strategy be effective. Very hard to know which behavior is appropriate in advance though. An assert that turns out to represent a benign circumstance that is already handled, multiplied by large numbers of machines, can be pretty annoying.
- waterhouse 6y agoYou have reached an imaginary program state. Please rotate your computer by 90 degrees and try again. (paraphrased from a friend's answering-machine message)
- saagarjha 6y agoI did that, but it's still a bit complex for me…
- js2 6y agoWhere are you quoting this from?
- jl6 6y agoDo we know if they have applied for the $500 bug bounty from DJB?
- kick 6y agoMy guess is that he'd never accept it. He originally didn't allow people to distribute qmail with changes; this is squarely a Debian issue.
- tedunangst 6y agoWhat did Debian change to introduce this vulnerability?
- dsr_ 6y agoNothing, per se. But they don't set up softlimits or message size limits by default, both of which you should definitely do... it's just that the values you put in are specific to your situation.
- tom_mellior 6y agoIf you have DJB's ego yelling at you that the default configuration is safe, why would you bother setting limits to fix it?
- jacquesm 6y agoBecause you care more about the users than about DJB's ego.
- haroldp 6y agoThen you should be running Postfix.
- dependenttypes 6y agoDJB has been telling people to run it with the memory limits since 2005.
- DominoTree 6y agoSerious question: Was there a legitimate reason they didn't patch this stuff when it was first discovered?
- dullgiulio 6y agoIt's not "they", it's Daniel J. Bernstein. That's the reason :) (If you don't know: he is a top cryptographer that can amazingly correct code. However, he also has a very big ego...)
- DominoTree 6y agoI just found this on Wikipedia where he raised his bug bounty and at the same time cited these "unexploitable" bugs https://en.wikipedia.org/wiki/Qmail#Security_reward_and_Georgi_Guninski's_vulnerability https://en.wikipedia.org/wiki/Qmail#Security_reward_and_Geor...
- stevekemp 6y agoqmail was essentially unmaintained for a long time, people were distributing patches to it, but there were no upstream releases. (Similar story with djbdns/tinydns.) "Recently" he released his code with new licenses, so that people could finally start distributing updated versions, rather than the previous approach where lots of people were sharing conflicting patches for various features (e.g. IPv6 support for AAAA records in tinydns.)
- rurban 6y agonotqmail.org is for long the defacto maintained version. They had this fixed long time ago.
- koheripbal 6y agoI don't understand. Most people with a big ego would not want a critical vulnerability associated with them. Can you elaborate?
- 6y ago
- upofadown 6y agoThe surprising thing here for me was that you can send qmail a 4GB email message and it will not immediately reject it in the default configuration.
- gowld 6y ago4GB was impossible when qmail was last updated.
- wahern 6y agoAccording to Wikipedia DJB's last release was 1998. 64-bit Alpha and SPARC V9 were in widespread use by then. Even Linux was ported to Alpha in 1995.[1] See https://www.linuxjournal.com/article/1044 https://www.linuxjournal.com/article/1044 [1] Actually, Jim Paradis announced the Linux/Alpha development kit January 23, 1995 on comp.os.linux.announce (Message-ID: 3g0787$59i@kruuna.Helsinki.FI). So it's possible he had the kernel ported in late 1994. He announced a fully functional distribution July 1, 1995. (Message-ID: 3t34ph$mi@kruuna.helsinki.fi)
- deleted 6y ago[deleted]
- tjalfi 6y agoIt's been years (2009) since I administered a mail server but I think this is also true of Sendmail.
- jedberg 6y agoDJB makes great code, but it always leaves a sour taste to use his code, because he has such a big ego and refuses to admit when he makes a mistake. He can't admit that it's possible to be the best but still make mistakes. I really hate supporting that.
- linsomniac 6y agoI once was talking to one of the household-name DNS guys and asked what he thought about djbdns and he said something along the lines of: "It's unfortunate that he's so abrasive, because he has some really great ideas that nobody will listen to because of it." I've more recently come a little bit to terms with this whole abrasive thing being a brain chemistry artifact, and not just being an asshole for assholes sake. Because it seems to disproportionately impact those of us in the technical field. Not to excuse it, but to understand it.
- SmallPeePeeMan 6y ago> Not to excuse it, but to understand it. Indeed. I don’t have to abide such people, and neither do you. Now if he had found a cure for cancer, would we have more tolerance?
- ergothus 6y agoI'm not a psychologist so I should probably shut up, but I'll continue: It seems like it does hit the tech field more often, yet overwhelmingly I see leading people in tech get softer, more empathetic, and more open to outside ideas the longer they work and the more experience they gain. Perhaps this is selection bias (the people I see are the "popular" ones, so of course they are the ones with more social awareness), or perhaps there is no hard and fast "genius requires ego". Note: I'm aware you didn't say that it did, you only mentioned that abrasive-ness seems disproportionately frequent in the tech field. I'm all for improving our understanding and ways of communication - I've definitely known people in the "asshole" category that were surprised and bothered to discover that was how they came across - but I'm not interested in normalizing abuse as a necessary cost. Which, again, you didn't say nor imply, but the concepts are related to what you've brought up, so I mention it here because the topic is interesting, not as a counterpoint.
- mmm_grayons 6y agoThis is why you don't run software that fossilized back in the nineties. Qmail was solid code but hasn't received the support and on-going development it needed. Use postfix instead for a secure MTA.
- smokecab 6y agoYour prejudice against out-of-fashion software is unfounded. Qmail had a bug because qmail had a bug, not because it hasn't been rewritten in the past week. The major web browsers, for example, are all under very active development, yet new RCE (!) bugs are regularly introduced into them.
- loup-vaillant 6y agoThe bug was known 15 years back, and not fixed.
- Snelius 6y ago"But we need to rewrite it with Rust!!" Crazy..
- mmm_grayons 6y agoI said nothing about rust, and the alternative I proposed (postfix) is written in C.
- mmm_grayons 6y agoThis isn't "out-of-fasion prejudice". This is a bug not being fixed for over a decade, which is clear evidence that it's not being adequately maintained. I've got nothing against qmail in principle, but it clearly hasn't received the attention it needs.
- titzer 6y agoInteger overflow bug causing manual memory allocation bug leading to RCE. All too common. We need to stop relying on software written in unsafe languages.
- foota 6y agoI would bet a lot of upvotes on this come from people misreading this as Gmail as I did :)
- gerdesj 6y agoNo. I ran several Qmails for many years on Mandrake with daemontools to supervise. All compiled from source. Lovely. "Life with Qmail" is a classic. I needed some additional flexibility and switched to Exim for my MTA of choice instead but have fond memories of Qmail.
- antirez 6y agoThat's quite interesting because the bug was there and was quite clear, and was not fixed by one of the most brilliant security persons out there (DJB). Maybe here the attitude really was the wrong one, considering how security oriented qmail was since the start, because the simplest possible fix does not seem to demand any other dimension, like complexity or performances.
- dependenttypes 6y agoWhile I think that he should have rewarded the original 2005 bug report I do not think that he did anything wrong after that. He was pretty clear about the memory limit for years now.
- ptx 6y agoIf the program only runs correctly with a memory limit in place, perhaps it should (if this is possible?) check for a memory limit when it starts and abort if it's missing.
- pwdisswordfish2 6y ago"Finally, we also discovered two minor vulnerabilities in qmail-verify (a third-party qmail patch that is included in, for example, Debian's qmail package): CVE-2020-3811 (a mail-address verification bypass), and CVE-2020-3812 (a local information disclosure)." The vulnerabilities were introduced in a third party patch, not source code that was authored or advocated by djb. "As recommended by Daniel J. Bernstein, qmail can be protected against all three 2005 CVEs by placing a low, configurable memory limit (a "softlimit") in the startup scripts of all qmail services." Seems much simpler than patching, let alone trusting someone else's patches. Interesting that they developed their exploit against Debian's default configuration. qmail from its source at https://cr.yp.to/qmail.html https://cr.yp.to/qmail.html comes with no such default configuration.
- mcguire 6y ago"About our new discovery, Daniel J. Bernstein issues the following statement: ""https://cr.yp.to/qmail/guarantee.html https://cr.yp.to/qmail/guarantee.html has for many years mentioned qmail's assumption that allocated array lengths fit comfortably into 32 bits. I run each qmail service under softlimit -m12345678, and I recommend the same for other installations.""
- loup-vaillant 6y ago> The vulnerabilities were introduced in a third party patch Which patch are you talking about? I don't see anything about a patch in the CVE. > Seems much simpler than patching, let alone trusting someone else's patches. We can patch the issue for good (and with a simple patch, apparently), or we can rely on users not to mess with seemingly unrelated init scripts. Do you trust third party users more than a (likely heavily scrutinised) small third party patch?
- pwdisswordfish2 6y agoqmail-verify. It is an "update" of the third party "realrcptto" patch by another third party. It says "third party patch" right there in the sentence I quoted. There is no qmail-verify in original qmail. As to the second question, I trust softlimit from daemontools and that is what I use. https://cr.yp.to/daemontools/softlimit.html https://cr.yp.to/daemontools/softlimit.html Not sure what "messing with seemingly unrelated init scripts" means. Can you be more specific?
- mftest 6y agoDoes anybody still use qmail in Production. I see on DJB's website that it is used in 700K sites but I doubt that assumption. Also the code has not been updated in almost a decade. How or why do sysadmins decide to go with qmail in 2020
- hackcasual 6y agoHad a friend of mine just unrack an apple server preparing for a data center move.
- ansible 6y agoYou can use a heap of patches for qmail, but I wouldn't recommend it. I was using Courier MTA for a while for self-hosting email. I may do so again, and will likely use mailcow or mail in a box. Neither are based on qmail.
- deleted 6y ago[deleted]
- xvilka 6y agoOne thing that might help to improve overall qmail code quality (even though it's already written by a genius of our time) is to rewrite qmail in Rust[1]. [1] https://github.com/notqmail/notqmail/issues/115 https://github.com/notqmail/notqmail/issues/115
- peterwwillis 6y agoThe only way that could improve the quality is for DJB to rewrite it in Rust. If anyone else rewrote it in Rust, it would just contain different security bugs, and probably run worse.
- knbt 6y agoThe code is concise k&r c, converting it to any language could be an informative experience
- jkuperus 6y agoThis is great. I remember when this came out. Georgi Guninski was a very accomplished bug hunter. Well known for finding vulnerabilities in web browers (ie. internet explorer) But the security community frowned upon that a bit as "not real hacking" so he was like "hold my beer", went after the hardest "real" target you could find and this was the result. The guy is a legend
- myrond 6y agoMaybe this will be useful to someone adding the following flags LDFLAGS='-z relro -z now' help with one of the holes (they mentioned it, but not what flags) Check out: https://github.com/slimm609/checksec.sh https://github.com/slimm609/checksec.sh * System-wide ASLR (kernel.randomize_va_space): Full (Setting: 2) * Does the CPU support NX: Yes * Core-Dumps access to all users: Restricted COMMAND PID RELRO STACK CANARY SECCOMP NX/PaX PIE FORTIFY # checksec --proc-all |grep qmail qmail-injectXXXXX27 Full RELRO Canary found Seccomp-bpf NX enabled PIE enabled Yes qmail-queueXXXXX97 Full RELRO Canary found Seccomp-bpf NX enabled PIE enabled Yes qmail-sendXXXXX78 Full RELRO Canary found Seccomp-bpf NX enabled PIE enabled Yes qmail-lspawnXXXXX89 Full RELRO Canary found Seccomp-bpf NX enabled PIE enabled Yes qmail-rspawnXXXXX90 Full RELRO Canary found Seccomp-bpf NX enabled PIE enabled Yes qmail-cleanXXXXX91 Full RELRO Canary found Seccomp-bpf NX enabled PIE enabled Yes