5 ms·
It's an SMTP server, started just two or three years ago, written by people who care about security and hope for it to be included in OpenBSD, written in... C.
by jerf 11y ago
It's an SMTP server, started just two or three years ago, written by people who care about security and hope for it to be included in OpenBSD, written in... C.
(Edit: Corrected by vezzy-fnord, started in 2009.)
The only currently-extant memory-unsafe language.
I... just... argh... when we will finally be rid of this albatross? What hope is there for the world of computer security if we can't even greenfield a solution to what has already proved to be one of the most dangerous security domains there is (thanks, sendmail) in something other than C? Why are the words "buffer overflow" appearing in relation to such young code? What am I doing with my life? Why am I even trying to write secure code in a world where this is even remotely considered an option?
- vezzy-fnord 11y agoThe only currently-extant memory-unsafe language. Some variants of Forth? OpenSMTPD actually dates back earlier to around 2009, when it was introduced in 4.6. Not that this is even notable, as virtually all major OpenBSD projects are C-based.
- nickpsecurity 11y agoYeah, there are other memory-unsafe languages. His overall point remains. Matter of fact, thanks to you, I can now drop bombs on concurrency angle thanks to Concurrent Pascal, Solo, etc. Relevant given all the C & BSD apps juggling threads. To think mainstream is just now solving these problems in past 3-5 years despite proven solutions available... (rolls eyes)
- vezzy-fnord 11y agoI don't see what concurrency has to do with any of this. Concurrent Pascal was an interesting project for its time, heavily dated by now. The greatest victory for concurrency was having architectures with thread-level parallelism instead of the dominant instruction-level parallelism, and one specifically optimized for CSP in the occam language. This was, of course, the transputer. So much for that. Erlang/OTP leads in the modern era. But I don't see any relation to our current topic of discussion.
- nickpsecurity 11y agoServer apps, including mail, implement forms of concurrency to handle incoming requests. Smart ones use either multi-threading or event model for performance. OpenBSD and others have all kinds of such apps they built on C. There have been numerous concurrency-related errors in major server apps due to this. So, just like with C itself, ignoring the safe, concurrency approaches from languages like Concurrent Pascal (or SCOOP/Eiffel today) is a failure to learn from the past with a direct, proven cost. Think all the race conditions, deadlocks, etc that have hit server apps would've happened on Solo, Ada tasking, or SCOOP? Probably fewer to none. So, it's still relevant and Ada/Eiffel are actually building on it. Especially SCOOP R&D community.
- vezzy-fnord 11y agoOh, you're criticizing shared-state concurrency, I take it. Event loops seem to have been the norm for C projects for a while. Both OpenSMTPD and qmail seem to rely on concurrency through forkexec, which would entail deferring to the kernel's CPU scheduling and job control.
- nickpsecurity 11y agoYeah, specifically unsafe handling of it. Forkexec seems to be a heavyweight (and less portable) approach that even DJB defended in his paper. My gripe should apply to any using threads. "Event loops seem to have been the norm for C projects for a while." I'll take you word for it as I've seen that in the few, top-tier ones I scrutinized. There's a similar trail of research dealing with that alternate model. Have a feeling the meme might apply there, too, but won't accuse them until I see for sure. ;)
- jerf 11y agoStarting in 2009 mitigates my criticism by, oh, let's say, 40%. Which I mean as an acknowledgment of your correction, even as I'm not generally backing down from the core point. I want to respect OpenBSD. To a great extent, I do. And there are reasons to prefer tested C codebases to untested code bases in new languages, sure. But to the extent I can't give my full, unadulterated respect to OpenBSD, it is because I don't see that they consider everything being in C a problem to be solved. I mean, I get engineering considerations and that we can't rewrite everything at the drop of a hat and performance matters, etc. But it is my opinion that any engineer that cares about security really ought to see "written in C" as starting out with a huge entry on the debit side, and take appropriate actions. And every year that goes by that's more true.
- vezzy-fnord 11y agoI'm conflicted on the issue. As a general principle, writing security-conscious code in C should be avoided. On the other hand, the issue is a tad more subtle. There seems to be an almost Pavlovian revilement and panic against C code, and this usually comes from people who haven't dealt with the language in a long time or at all in any depth. For one thing, the blanket assertion that secure C code is a Herculean task, doesn't quite pan out. djb showed one way, based on a few general principles espoused in qmail, that has since successfully been reapplied by people like Wayne Marshall, Laurent Bercot, Bruce Guenter, Paul Jarc, JdeBP and so on. Much of it comes from abandoning the C standard library in favor of more robust alternatives like libdjb variants, clearly architecting process/module boundaries, using simple formats like netstrings and text0 that need not be "parsed" in the conventional sense, etc. Further along the line, we have the SEI CERT C Coding Standard that comprehensively identifies many pitfalls. [1] Deeper still, cryptography is one area where abstraction can be an enemy. You actually have to deeply worry about time complexity, branch prediction, precise timing, compiler optimizations and so on [2]. A tool like C becomes desirable here. Lastly, but crucially - you're on Unix, for better or for worse. POSIX, SUS, all of those standards? They're deeply written with the conventions and API/ABI semantics of C in mind. Such is life. When you primarily write code at that level, the easiest way not to have any impedance mismatch is to just write it in C. [1] https://www.securecoding.cert.org/confluence/display/c/SEI+CERT+C+Coding+Standard https://www.securecoding.cert.org/confluence/display/c/SEI+C... [2] https://cryptocoding.net/index.php/Coding_rules https://cryptocoding.net/index.php/Coding_rules
- nickpsecurity 11y agoI agree. Further, these problems have been known since the 1960's with solutions available then. Should've been adopted once the cheap hardware caught up in performance. The thing that surprises me is how many people think C's design decisions were well-thought or something vs competition. No, they just stripped every good protection or feature from ALGOL, etc so it would fit on the PDP-11. Did same for UNIX. That made sense given what they had on hand but keeping it all the way into 2015? See main article's gripes for entirely-predictable result. Gave details here on C and UNIX history with counterexamples from other systems: https://news.ycombinator.com/item?id=10273656 https://news.ycombinator.com/item?id=10273656
- rhelmer 11y agoI am hoping that Rust will make in-roads into the BSD world, since it interoperates well with C and is much safer by default while not imposing overhead like a managed language+runtime would. It also has very liberal licensing (GCC being GPL has always been a tough pill for some to swallow, FreeBSD at least has already switched to Clang) https://github.com/rust-lang/rust#license https://github.com/rust-lang/rust#license The OpenBSD folks have done great work through auditing and writing safer code in C, and I agree that using a language that ameliorated at least some of the common mistakes programmers make would be a net positive. Related - this blog from @tedunangst is worth reading, regarding the limit to which even a language like Rust can protect against specific exploits (the OpenSSL "Heartbleed" vulnerability, in this case) http://www.tedunangst.com/flak/post/heartbleed-in-rust http://www.tedunangst.com/flak/post/heartbleed-in-rust
- jerf 11y agoWhat tedunangst says is true enough in the logical sense, but it makes me nervous in the human sense. Humans seem prone to the following logic failure: 1. Object X frequently removes limbs from the user. 2. Object Y does the same thing, but more safely. 3. Wait, Object Y causes hangnails if you misuse it hard enough. 4. It's OK to keep using Object X! Like many fallacies, this doesn't survive being clearly written out, but I still see it an awful lot.
- rhelmer 11y agoI don't know, I think it's a pretty fair treatment - there are links to an advisory that Ted thinks Rust could have prevented (http://www.x.org/wiki/Development/Security/Advisory-2015-02-10/ http://www.x.org/wiki/Development/Security/Advisory-2015-02-...) and to a blog post disagreeing with his http://tonyarcieri.com/would-rust-have-prevented-heartbleed-another-look http://tonyarcieri.com/would-rust-have-prevented-heartbleed-...
- jerf 11y agoI am saying tedunangst's treatment is fair. What I don't like is what people in general tend to do with the argument. I am not claiming tedunangst is doing that with the argument.
- sucramb 11y agoLinux Kernel, Apache, nginx. All written in C. You are trolling.
- jerf 11y agohttp://osvdb.org/search/search?search[vuln_title]=buffer+overflow&search[text_type]=titles&search[refid]=&search[referencetypes]=&kthx=search http://osvdb.org/search/search?search[vuln_title]=buffer+ove... To a first approximation, every single one of those wouldn't happen in a memory-safe language. Of course, plenty of other things will. But it's not like choosing memory-unsafety makes those other things less like (memory unsafety does not, for instance, prevent incorrect ACL specification), so that's not very compelling. Think of it this way. Suppose C didn't exist, and somebody proposed a new language today that was memory unsafe. Would it have any success, or be laughed out of existence? In fact, go ahead and suppose C did exist, because it would still not be taken seriously. If it's so wonderful that C is memory-unsafe, where's all the new languages that are also memory-unsafe? (Note having specially-called out "unsafe" operators does not make a language "unsafe" the way C is.)
- nickpsecurity 11y agoA few big projects written in C proves nothing about them or C. Need the larger picture with many instances to prove a claim. Certain, preventable vulnerabilities have occurred mostly in C software. Another commenter gave an example. Many of these similarly occurred in the C-lang projects you reference. So, using C instead of a language that prevents them was the first reason for their existence. C being a plague still stands.
- DominikD 11y agoIt doesn't, it's a fallacy. A lot of code that we think of today as buggy was written in the era in which people did not understanding threats and consequences of certain decisions. The reason we have nice things today is that we were lucky enough to have languages like C to experiment on and with and learn. "Safe" languages of today have the benefit of hindsight that good old C didn't have. Comparing legacy code in C to modern code in whatever you deem worthy is not only unhelpful but also very deceptive. Compare new to new and tell me: how many web, smtp, imap, dns servers in Python, C# or Rust do we run today? There were some attempts but very few successful ones, despite of all the goodness that comes with better languages, libs and tools. I bet there's a reason behind it, other than just "people didn't try hard enough". There's old stuff out there being used today that we should definitely phase out. But we'll replace system software with new one written in system languages. Most likely C, perhaps Rust. But definitely now with ones written in "safe" languages that should deliver (if they are so awesome) but somehow didn't.
- zokier 11y agoAbandoning C would mean abandoing one of the very cornerstones of UNIX, or even the heart and soul of UNIX. While I won't be shedding many tears over that, I do expect OpenBSD to remain one of the last fortresses of C instead of being in the forefront of abandoning it. It it their way of doing things and they have showed little interest for appeasing the masses by changing themselves. So while I don't like C (or UNIX for that matter) I still do celebrate the fact that these people are actually doing something and even more importantly they are doing what they want despite the naysayers and doing it well.
- stsp 11y ago"The OpenBSD project produces a FREE, multi-platform 4.4BSD-based UNIX-like operating system." You can't really divorce UNIX from C. The only other language option for an smtpd server in OpenBSD base would have been Perl, and I bet people would complain about that just the same. And perhaps we wouldn't be able to review it properly if it was written in Perl. Perhaps Marc Espie could. But OpenBSD has plenty of people on the team who are well trained at auditing C. I'll be doing my part and read some smtpd code during an 8h train ride tomorrow. I already read one file and found one bug in it and told gilles about it. That's how this works. Changing the language won't magically fix anything.
- zx2c4 11y ago> I'll be doing my part and read some smtpd code during an 8h train ride tomorrow. I already read one file and found one bug in it and told gilles about it. Wonderful! This is exactly what I (and many others) were hoping for. Thanks so much for your willingness to jump in like that.
- stsp 11y agoDid you try to tell gilles, eric, and some other committers about your concerns in private, before posting on a public mailing list and then submitting your own post to HN?
- zx2c4 11y agoIt's a conversation that's been going on for a long time now over various channels but has/had mostly stagnated into a disappointing lull of complacency. The Quayls results were a big wake up to that, and my own finding yesterday was the final straw: fresh ideas and voices on how to carry the project toward maturation are quite necessary, as well as smart folks like yourself becoming interested in reviewing the source code.
- poolpOrg 11y agoNo, he didn't. As much as he'd like to make it like we don't fix bugs, our public bug tracker is one click away and shows that out of the 25 open issues and the 493 closed issues, none mentions even remotely a pending crash that he is experiencing. He does have some feature requests and bugs though, with many handled already, so it seems that he does knows where to find it when he needs to. As for his concerns, yes he did tell me, he sent me 16 mails yesterday, including the one with the trace. Unfortunately, he sent it past midnight, at about the same time he made it public because he just couldn't wait until morning to disclose.
- DominikD 11y agoThis is a typical HN response: write system code in a hip, modern language - this will solve all of the problems. Which language? I haven't seen a single successful, commercial OS written in something else than C or C++. Part of this is the amount of legacy code, part of it is compiler and stdlib maturity, performance and predictability. You can probably get maturity with some of the existing "safe" languages, but rarely would you get any of the other guarantees. If you can throw HW on the problem, you can probably get away with SMTP server in, say, Python. But you typically can't do just that.
- glass- 11y ago> Which language? Haven't you been paying attention? You're supposed to write your security-critical code in a language that was released 5 months ago and has no major deployments yet.
- DominikD 11y agoThis view is too cynical even for me. ;)
- alricb 11y agoI thought we were supposed to use Concurrent Pascal? Did I miss a memo?