10 ms·

Introducing OpenBSD's new httpd [pdf]

197 points by fcambus 12y ago

fcambus 12y ago
Slides are available here : http://www.openbsd.org/papers/httpd-slides-asiabsdcon2015.pdf http://www.openbsd.org/papers/httpd-slides-asiabsdcon2015.pd...
detaro 12y ago
Supports TLS using LibreSSL, serves static files and FastCGI. https://github.com/reyk/httpd/issues?q=label%3Afeaturitis+is%3Aclosed https://github.com/reyk/httpd/issues?q=label%3Afeaturitis+is... featuritis tag in die bugtracker for currently denied features. Clearly aiming for as simple as possible while being useful.
- reidrac 12y ago
  More information: http://bsd.plumbing/about.html http://bsd.plumbing/about.html
- bluetech 12y ago
  What is wrong with compressing static files?
  - falcolas 12y ago
    That's a good question. Though a better question might be "why not serve pre-compressed static files alongside regular files?" Compression will change your server from being network throttled to being CPU throttled. And in this day and age, we can scale CPUs more easily than we can bandwidth.
    - bluetech 12y ago
      Pre-compressing is possible, but why shouldn't the http server do it for you? The server can cache the compressed files in memory and/or disk as it sees fit (although gzip was pretty cheap last time I checked). It can also deal with cache invalidation better than humans - I speak from experience :).
      - falcolas 12y ago
        Very true. The difference lay in the benefit of not having to implement the compression logic as part of the server, making the server implementation much simpler.
  - MichaelGG 12y ago
    And it's funny, Windows/IIS makes you install dynamic compression separately, in the name of security. (Course, they also make you install the tftp client separately, under the same "reasoning".)
- Gracana 12y ago
  In one of the comments on the "featuritis" tagged issues: > I add the label "featuritis" to remind us of extra features (eg. ldap) that we reject now but might want to reinspect later. So it's not that httpd will never be extended beyond the basics, but these issues are simply out of scope right now. I like that approach.
jacquesm 12y ago
I've been going through the code for the last half hour and I really hope this isn't representative of what the OpenBSD group considers to be defensive C programming. Stack allocated buffers, questionable logic and a generally terrible style as well as a complete lack of comments. Don't take my word for it, see for yourself: https://github.com/reyk/httpd/blob/master/httpd/server.c https://github.com/reyk/httpd/blob/master/httpd/server.c The "new" is a bit off too, the copyright runs 2006-2015.
- jedisct1 12y ago
  I don't see anything wrong with that code. There is nothing wrong with using the stack for what it was designed for. The function names are self-explanatory. The "style" might not be yours, but it doesn't make it bad. And as described in the slides, it was derived from relayd.
  - jacquesm 12y ago
    > I don't see anything wrong with that code. Look longer. Let me give you some examples: Forward declarations for functions that could be avoided by re-arranging the code, using both 0 and -1 to indicate error returns from functions, bits like: s = fd == -1 ? socket(ss->ss_family, SOCK_STREAM, IPPROTO_TCP) : fd; if (s == -1) goto bad; Sure, I can read that but it takes more effort than it should and could be re-written much clearer and so on. > There is nothing wrong with using the stack for what it was designed for. The stack should not be used to allocate buffers intended to hold data written by other routines or read from untrusted sources for reasons that have become painfully clear over the last couple of years. > The function names are self-explanatory. Strong disagree about the function names being self explanatory, plenty of the functions have non-obvious side-effects. The 'style' is asking for trouble and as for it being derived from 'relayd' that pretty much confirms that this isn't something new (which is actually a good thing), but an adaptation of something old to a new role (such adaptations are security wise something to be very wary of, re-purposing old code is a great way to find out what edge cases were missed previously). Far more symbols are exported than necessary. > The "style" might not be yours, but it doesn't make it bad. There's a return at the end of a function returning void for no reason. #if 0'd old code that should simply be purged. In the server_log code I think there may be a path to get a double free of 'ptr' where it is used first in the block with the while loop, then not reset to NULL and re-used in the second block and freed if it is not NULL (which it still is from the previous block...). The style used obscures this possibility.
andor 12y ago
FastCGI: The protocol provides the single and fast interface to serve dynamic content That's a bad choice in my opinion. Without reverse proxy functionality httpd can't match the flexibility of nginx.
- jedisct1 12y ago
  This is needed at least for PHP. Granted, PHP has a built-in web server, but it's really just for testing and not meant to be a replacement for fpm. Maybe reverse proxying is coming up next, as well as HTTP2. But once again, the goal is not to have a full-featured server that can replace Nginx. Rather something small, simple and secure.
- twic 12y ago
  relayd does reverse proxying in OpenBSD: https://calomel.org/relayd.html https://calomel.org/relayd.html So there's no need for httpd to do it as well.
  - pyvpx 12y ago
    please do not link to calomel.org. OpenBSD takes great care to have clear, concise, accurate documentation available both on the system and on the web. link to the man page! calomel.org is filled with...ignorance, to say the least.
    - twic 12y ago
      I had a look at the relayd man page, but it's not actually very useful. I wanted a document that would demonstrate the use of relayd as an http proxy server. The man page doesn't even mention http. If you would like to suggest a better reference, i will happily edit my post.
- knivets 12y ago
  Why is it a bad choice? Why does it need reverse proxy functionality? Why not use some reverse proxy software (e.g. HAProxy) in order to follow the UNIX modular principle? I'm not trying to argue here, but rather trying to understand.
vacri 12y ago
Why use a name that's already in use as a general descriptor? At least the other httpds have names that can be used to differentiate them: http://en.wikipedia.org/wiki/Httpd http://en.wikipedia.org/wiki/Httpd
- ape4 12y ago
  Yeah, it needs a real name.
- wongarsu 12y ago
  In practice, everybody will just say "OpenBSD httpd" and there will be little to no name confusion.
  - vacri 12y ago
    Ugh, there's 10 syllables in that. Maybe they should call it o10d?
- deleted 12y ago
  [deleted]
xiaq 12y ago
So... Apache was removed from base on Mar 14 2014 in favor of nginx, and nginx on Aug 27 2014 in favor of OpenBSD httpd. For sysadmins who closely follow the "recommended" way, having to migrate the configurations of the http server twice within half a year must have been a frustrating experience. Also, I wonder what "removal from base" means exactly - can you still install them (the OpenBSD-patched versions) from the ports collection or something like that?
- sarnowski 12y ago
  That's correct. They are still available in the ports. If were operating a complex setup, you probably already used nginx from ports before. Not being in the base means that it doesn't get the same security attention as its not officially part of OpenBSD anymore.
  - xiaq 12y ago
    > If were operating a complex setup, you probably already used nginx from ports before. Does "before" refer to the period before Apache was dropped from and nginx was imported to base? (Since apparently you can just use nginx from the base when it is there.) EDIT: It just occurred to me that "complex setup" likely indicates that you would like to change some build flags, so you have to install from the ports. Stupid me.
  - brynet 12y ago
    And the nginx in ports still maintains at least one significant OpenBSD patch: chroot(2) functionality.
- 4ad 12y ago
  Apache was removed from base in 2014, but nginx was added to base in 2011. Sysadmins had ~2 years to migrate from Apache to nginx (and if they didn't want to migrate, they could continue to use Apache from ports). For nginx, they only had 6 months to migrate (though again, they can still use nginx from ports). So no, they didn't have to migrate the http configuration twice a year, more like twice in 4 years, although, considering that OpenBSD only ever included into base 3 web servers, more like twice in 16 years.
ezequiel-garzon 12y ago
If I may take this opportunity... Does anybody know what I'm supposed to put in /etc/ssl/server.crt for SSL encryption? I have concatenated all six possible permutations of my own certificate ssl.crt, the intermediate certificate sub.class1.server.ca.pem and the root certificate ca.pem, but this gives me the error The certificate is not trusted because no issuer chain was provided. (Error code: sec_error_unknown_issuer) (my Ubuntu Chrome gives me a green lock, though). Feel free to visit my blank site https://ezequiel-garzon.net https://ezequiel-garzon.net Thanks!
- icebraining 12y ago
  Works fine here. Where are you getting an error? Maybe that client doesn't have StartCom's root cert in its trust store.
  - ezequiel-garzon 12y ago
    Thanks for your reply. Ubuntu Firefox, Android Chrome... scary warnings pretty much everywhere. It'd be surprising if all these clients didn't have StartCom's root certificate. If you don't mind, how did you set up your /etc/ssl/server.crt file?
    - phaer 12y ago
      Are you sure they have the same root certificate? About a month ago, I ran in some problems because I got a new certificate from RapidSSL which had been signed by their new-ish SHA256 certificate but older clients (~3 years old browsers) had only RapidSSLs old root certificate installed.
      - ezequiel-garzon 12y ago
        Thanks for answering. I'm not sure, but wouldn't this be a widespread issue, given the popularity (read free as in beer) of StartCom certificates? I'm pretty sure I'm doing something wrong.
floatboth 12y ago
Why do they even have an httpd in base? They like to say they're smaller and simpler than FreeBSD, but FreeBSD doesn't include a web server in base!
- clarkevans 12y ago
  It's very handy to wrap services in HTTP to expose them outside the machine, limiting outside access to a single port with clear FastCGI delegations to specific functions. If you don't build this in, then services shouldn't assume it to be operational and can't auto-configure themselves.
- stsp 12y ago
  One reason is that network installs require http. The base system would be unable to self-replicate across network installs without httpd.
  - floatboth 12y ago
    Why not FTP? Or even TFTP?
    - pyvpx 12y ago
      you can do that as well.
    - tedunangst 12y ago
      FTP is dead.
      - wglb 12y ago
        How I wish this were true.
        protomyth 12y ago
        Yeah, don't you just love how "Enterprise" vendors make it more secure with going with ftps instead of picking a different protocol?
joosters 12y ago
OpenBSD seems to have caught a bad case of the 'not invented here' sickness. If they didn't like where nginx was going, why not just fork it and have a working web server with a known codebase? The forks would diverge but they could still grab fixes from nginx whenever they wanted to.
- clarry 12y ago
  See how well forking has worked in the past? Especially with large components like compilers or web servers (see old apache, old gcc). Such forks end up antiquated and more or less stagnant. Besides, grabbing fixes just like that doesn't work so well once you've diverged enough and the upstream is constantly growing and changing. RTFA and you'll see this point discussed.
- stsp 12y ago
  This wasn't a question about having a functional webserver or not. It was a conscious decision against running code without reading it first. The "run but don't read" strategy belongs into ports, not base. Hence nginx moved to ports. Forking nginx was actually discussed when a proposed nginx update diff was too large for proper review. Tons of complex regex parsing code was added with nobody willing to go through it all in detail. The forking option was quickly dismissed. The tipping point happened when someone (reyk?) pointed out that relayd had most of the guts of a complete web server anyway, including OpenBSD-style privsep which has to be bolted on to virtually all software imported from elsewhere, including nginx. It took about a week from that discussion happening to having a relayd-based functional and peer-reviewed web server in base (+ a couple of months of shaking out a few bugs and adding minor features).
  - jimktrains2 12y ago
    > he "run but don't read" strategy belongs into ports, not base. Hence nginx moved to ports. Why is there a webserver in base?
    - jquast 12y ago
      For the same reason the other 20+ net-daemons are: http://cvsweb.openbsd.org/cgi-bin/cvsweb/~checkout~/src/etc/rc.conf?content-type=text/plain http://cvsweb.openbsd.org/cgi-bin/cvsweb/~checkout~/src/etc/... With OpenBSD, you can actually avoid the ports tree all together and still play a critical role in your infrastructure. If you're going to distribute a system that doesn't do much of anything on its own, and delegate the responsibility of security (and choice!) to "the ports tree" you're not doing any better than Linux or FreeBSD.
cturner 12y ago
What would be the elegant way to implement websockets on the new openbsd arrangement? Would it be to use relayd instead of httpd? Or is websocketd suitable for the openbsd base?
- detaro 12y ago
  Probably to install nginx from the ports and use that. I don't think supporting web sockets would fit the scope of the base system.
zx2c4 12y ago
I love these guys: https://github.com/reyk/httpd/blob/master/httpd/server_http.c#L802-L804 https://github.com/reyk/httpd/blob/master/httpd/server_http....
- fredsted 12y ago
  Hmm, Comic Sans, weird choice for an error message.
- fafner 12y ago
  At least they added Comic Neue. I'm a bit disappointed that a project like OpenBSD that is so vocal about free software is promoting a proprietary font like Comic Sans MS.
  - termie 12y ago
    Not sure if whooshed, but just in case anyone else doesn't get it, you're not going to see Comic Sans unless you load it from a Windows machine or a Unix distribution that's installed the font. IOW, it's just a gentle jab at Microsoft and perhaps those distributions that install the MS fonts. Technically that is promotion in name but not spirit.
    - fafner 12y ago
      Well the OpenBSD devs seem to use Comic Sans in their presentations though E.g., http://www.openbsd.org/papers/ru13-deraadt/index.html http://www.openbsd.org/papers/ru13-deraadt/index.html
      - termie 12y ago
        You're right, they use it as a weapon of snark and for fund raising: http://www.openbsd.org/papers/bsdcan14-libressl/mgp00025.html http://www.openbsd.org/papers/bsdcan14-libressl/mgp00025.htm...
    - setpatchaddress 12y ago
      You'll see it on the Mac, too, since the MS web fonts are installed by default.
jalfresi 12y ago
Does anyone know if the FastCGI implementation is complete i.e. it supports FastCGI processes in all three roles; Responder, Authorizer and Filter? I've always wanted to use FastCGI more but most implementations (in Apache and Nginx at least) only support some of those roles (or require work arounds using server specific features; i.e. apache filters rather than FastCGI filters)
- tedunangst 12y ago
  I think it's "it runs the software we tested with" complete.
mdekkers 12y ago
any performance benchmarks in the wild?
jnazario 12y ago
[update - i read the back story elsewhere and the reason is less boneheaded than i had assumed. still, i think the community needs to focus on higher priority needs and gaps] this is the sort of thing that makes me happy i'm no longer involved in the OpenBSD world. httpd & previously smtpd are two replacements that (in my opinion) have little additive value beyond existing, community-adopted solutions (e.g. nginx and postfix), diluting effort where it is needed. does the world need a new httpd? maybe. but the world needs other replacement software to be done first because it'll have a greater impact. for example, OpenBSD could invest time and effort in maturing static code analyzers to assist in code audits (especially of ports). i suspect this new httpd was done less because it was needed and more because it could be done. that's the attitude i disagree with.
- tedunangst 12y ago
  Not all developers are fungible.
  - deleted 12y ago
    [deleted]
- p9idf 12y ago
  If the world needed OpenBSD programmers to write high-impact replacement software so badly, the world would offer to pay for it. And since evidently no one wants to pay them to do so, it seems pretty reasonable to expect that the unpaid programmers -- volunteering their free time to write programs that they give away to the world for free -- would set their own priorities, address their own needs, and invest their time and effort however they see fit.
- floatboth 12y ago
  OpenSMTPD is much, much simpler than Postfix. I can't stand Postfix's fucking 1000 line multi-file configuration. Nginx is simple, so, yeah – no need for a replacement there.
  - ta0967 12y ago
marc_omorain 12y ago
Is there a technical reason why you would implement HTTPS in a HTTP server? If you ran a separate process on port 443 to terminate SSL connections, and then proxy that request to a HTTP server running locally, there would be better separation of concerns. For example, this setup would mean that a security flaw in the HTTP server that allowed a user to read memory would not be able to read any private keys used in the HTTPS server. I guess some downsides would be some extra latency while the request is proxied, and some extra memory overhead for the second process. I'm interested in anyones thoughts on this.
- Too 12y ago
  I'm not into the specifics of https but i've been writing a lot of gateways for other protocols and it's never as easy as "just forward the message". Protocols some times have state that the proxy must be aware of and sometimes the forwarding is conditional which means that the proxy must understand both types of protocol and be able to act based on information in it. Say for example you want to block all requests to a specific resource, if your https server knows about this you might be able to reject the request before you decrypt everything of it.
- throwaway2048 12y ago
  OpenBSD has added support into libressl for privilege separated processes that hold SSL keys, any operation requiring the use of private keys such as the creation of session keys, and signing things are shuttled off via a small api to a separate process. This is somewhat analogous to what ssh-agent does for openssh clients. OpenBSD's TLS private key consuming daemons have moved to this model or are in the process of doing so. This helps to mitigate the problem of access to process memory results in disclosed private keys, also the requirement of the daemon's user facing bits to have access to the keyfiles. http://article.gmane.org/gmane.os.openbsd.cvs/139527/ http://article.gmane.org/gmane.os.openbsd.cvs/139527/
- zurn 12y ago
  Proxying typically loses a lot of information, like client address, client certificate info and so on. If you take care to make carry over everything transparently, it would more accurately be called privilege separation (ala sshd).
amelius 12y ago
I'm amazed we still use configuration files like that. Why not have some capable programming language (python, javascript) handle the configuration? This would let one more naturally describe things, and will eventually reduce the possibility of obscure configuration commands that do one very specific thing, and hence lead to frustration when the user needs something slightly different. Using a programming language, one could even configure using callbacks, etc. and is much more flexible.
- m0th87 12y ago
  I'm not a lisper, but this makes me pine for some universal, maybe lisp-like language. It's a total waste of time to learn a new config syntax for each application.
  - gjggkjasdf 12y ago
    the ironic thing here is that Lispers fucking love sticking new DSLs wherever they possibly can. And I imagine most developers of these pet config files would argue the same thing. Unfortunately. It was Microsoft of the 1990s that surprisingly brought some sanity to the table, with INI files. I'm glad tools like Git use this format.
    - ams6110 12y ago
      Then they threw sanity out the window with the registry.
  - throwaway2048 12y ago
    This is getting pretty offtopic, but this is what GNU has tried to do with guile https://en.wikipedia.org/wiki/GNU_Guile https://en.wikipedia.org/wiki/GNU_Guile
  - catwell 12y ago
    Lua is often used for that purpose. It is not a Lisp but it fulfills this role well enough.
- 12y ago
davidgerard 12y ago
I read the config file format and I fell in love. I really hope this gets the portable treatment.
davidgerard 12y ago
I read the config file format and I fell in love. I really hope this gets the portable treatment.
captaintod 12y ago
Surprised that nobody has mentioned ashd yet: http://www.dolda2000.com/~fredrik/ashd/ http://www.dolda2000.com/~fredrik/ashd/
dyoder 12y ago
Did OpenBSD just standardize on an HTTP server they wrote in 2 weeks, has no tests, doesn't fully implement the spec…and then brag about it?
- tedunangst 12y ago
  No.
kymywho 12y ago
Https authentication support for Subversion could be the killer feature.

197 points by fcambus 12y ago

Slides are available here : http://www.openbsd.org/papers/httpd-slides-asiabsdcon2015.pdf http://www.openbsd.org/papers/httpd-slides-asiabsdcon2015.pd...

detaro 12y ago

Supports TLS using LibreSSL, serves static files and FastCGI. https://github.com/reyk/httpd/issues?q=label%3Afeaturitis+is%3Aclosed https://github.com/reyk/httpd/issues?q=label%3Afeaturitis+is... featuritis tag in die bugtracker for currently denied features. Clearly aiming for as simple as possible while being useful.

reidrac 12y ago
More information: http://bsd.plumbing/about.html http://bsd.plumbing/about.html
bluetech 12y ago
What is wrong with compressing static files?
- falcolas 12y ago
  That's a good question. Though a better question might be "why not serve pre-compressed static files alongside regular files?" Compression will change your server from being network throttled to being CPU throttled. And in this day and age, we can scale CPUs more easily than we can bandwidth.
  - bluetech 12y ago
    Pre-compressing is possible, but why shouldn't the http server do it for you? The server can cache the compressed files in memory and/or disk as it sees fit (although gzip was pretty cheap last time I checked). It can also deal with cache invalidation better than humans - I speak from experience :).
    - falcolas 12y ago
      Very true. The difference lay in the benefit of not having to implement the compression logic as part of the server, making the server implementation much simpler.
- MichaelGG 12y ago
  And it's funny, Windows/IIS makes you install dynamic compression separately, in the name of security. (Course, they also make you install the tftp client separately, under the same "reasoning".)
Gracana 12y ago
In one of the comments on the "featuritis" tagged issues: > I add the label "featuritis" to remind us of extra features (eg. ldap) that we reject now but might want to reinspect later. So it's not that httpd will never be extended beyond the basics, but these issues are simply out of scope right now. I like that approach.

jacquesm 12y ago

I've been going through the code for the last half hour and I really hope this isn't representative of what the OpenBSD group considers to be defensive C programming. Stack allocated buffers, questionable logic and a generally terrible style as well as a complete lack of comments. Don't take my word for it, see for yourself: https://github.com/reyk/httpd/blob/master/httpd/server.c https://github.com/reyk/httpd/blob/master/httpd/server.c The "new" is a bit off too, the copyright runs 2006-2015.

jedisct1 12y ago
I don't see anything wrong with that code. There is nothing wrong with using the stack for what it was designed for. The function names are self-explanatory. The "style" might not be yours, but it doesn't make it bad. And as described in the slides, it was derived from relayd.
- jacquesm 12y ago
  > I don't see anything wrong with that code. Look longer. Let me give you some examples: Forward declarations for functions that could be avoided by re-arranging the code, using both 0 and -1 to indicate error returns from functions, bits like: s = fd == -1 ? socket(ss->ss_family, SOCK_STREAM, IPPROTO_TCP) : fd; if (s == -1) goto bad; Sure, I can read that but it takes more effort than it should and could be re-written much clearer and so on. > There is nothing wrong with using the stack for what it was designed for. The stack should not be used to allocate buffers intended to hold data written by other routines or read from untrusted sources for reasons that have become painfully clear over the last couple of years. > The function names are self-explanatory. Strong disagree about the function names being self explanatory, plenty of the functions have non-obvious side-effects. The 'style' is asking for trouble and as for it being derived from 'relayd' that pretty much confirms that this isn't something new (which is actually a good thing), but an adaptation of something old to a new role (such adaptations are security wise something to be very wary of, re-purposing old code is a great way to find out what edge cases were missed previously). Far more symbols are exported than necessary. > The "style" might not be yours, but it doesn't make it bad. There's a return at the end of a function returning void for no reason. #if 0'd old code that should simply be purged. In the server_log code I think there may be a path to get a double free of 'ptr' where it is used first in the block with the while loop, then not reset to NULL and re-used in the second block and freed if it is not NULL (which it still is from the previous block...). The style used obscures this possibility.

andor 12y ago

FastCGI: The protocol provides the single and fast interface to serve dynamic content That's a bad choice in my opinion. Without reverse proxy functionality httpd can't match the flexibility of nginx.

jedisct1 12y ago
This is needed at least for PHP. Granted, PHP has a built-in web server, but it's really just for testing and not meant to be a replacement for fpm. Maybe reverse proxying is coming up next, as well as HTTP2. But once again, the goal is not to have a full-featured server that can replace Nginx. Rather something small, simple and secure.
twic 12y ago
relayd does reverse proxying in OpenBSD: https://calomel.org/relayd.html https://calomel.org/relayd.html So there's no need for httpd to do it as well.
- pyvpx 12y ago
  please do not link to calomel.org. OpenBSD takes great care to have clear, concise, accurate documentation available both on the system and on the web. link to the man page! calomel.org is filled with...ignorance, to say the least.
  - twic 12y ago
    I had a look at the relayd man page, but it's not actually very useful. I wanted a document that would demonstrate the use of relayd as an http proxy server. The man page doesn't even mention http. If you would like to suggest a better reference, i will happily edit my post.
knivets 12y ago
Why is it a bad choice? Why does it need reverse proxy functionality? Why not use some reverse proxy software (e.g. HAProxy) in order to follow the UNIX modular principle? I'm not trying to argue here, but rather trying to understand.

vacri 12y ago

Why use a name that's already in use as a general descriptor? At least the other httpds have names that can be used to differentiate them: http://en.wikipedia.org/wiki/Httpd http://en.wikipedia.org/wiki/Httpd

ape4 12y ago
Yeah, it needs a real name.
wongarsu 12y ago
In practice, everybody will just say "OpenBSD httpd" and there will be little to no name confusion.
- vacri 12y ago
  Ugh, there's 10 syllables in that. Maybe they should call it o10d?
deleted 12y ago
[deleted]

xiaq 12y ago

So... Apache was removed from base on Mar 14 2014 in favor of nginx, and nginx on Aug 27 2014 in favor of OpenBSD httpd. For sysadmins who closely follow the "recommended" way, having to migrate the configurations of the http server twice within half a year must have been a frustrating experience. Also, I wonder what "removal from base" means exactly - can you still install them (the OpenBSD-patched versions) from the ports collection or something like that?

sarnowski 12y ago
That's correct. They are still available in the ports. If were operating a complex setup, you probably already used nginx from ports before. Not being in the base means that it doesn't get the same security attention as its not officially part of OpenBSD anymore.
- xiaq 12y ago
  > If were operating a complex setup, you probably already used nginx from ports before. Does "before" refer to the period before Apache was dropped from and nginx was imported to base? (Since apparently you can just use nginx from the base when it is there.) EDIT: It just occurred to me that "complex setup" likely indicates that you would like to change some build flags, so you have to install from the ports. Stupid me.
- brynet 12y ago
  And the nginx in ports still maintains at least one significant OpenBSD patch: chroot(2) functionality.
4ad 12y ago
Apache was removed from base in 2014, but nginx was added to base in 2011. Sysadmins had ~2 years to migrate from Apache to nginx (and if they didn't want to migrate, they could continue to use Apache from ports). For nginx, they only had 6 months to migrate (though again, they can still use nginx from ports). So no, they didn't have to migrate the http configuration twice a year, more like twice in 4 years, although, considering that OpenBSD only ever included into base 3 web servers, more like twice in 16 years.

ezequiel-garzon 12y ago

If I may take this opportunity... Does anybody know what I'm supposed to put in /etc/ssl/server.crt for SSL encryption? I have concatenated all six possible permutations of my own certificate ssl.crt, the intermediate certificate sub.class1.server.ca.pem and the root certificate ca.pem, but this gives me the error The certificate is not trusted because no issuer chain was provided. (Error code: sec_error_unknown_issuer) (my Ubuntu Chrome gives me a green lock, though). Feel free to visit my blank site https://ezequiel-garzon.net https://ezequiel-garzon.net Thanks!

icebraining 12y ago
Works fine here. Where are you getting an error? Maybe that client doesn't have StartCom's root cert in its trust store.
- ezequiel-garzon 12y ago
  Thanks for your reply. Ubuntu Firefox, Android Chrome... scary warnings pretty much everywhere. It'd be surprising if all these clients didn't have StartCom's root certificate. If you don't mind, how did you set up your /etc/ssl/server.crt file?
  - phaer 12y ago
    Are you sure they have the same root certificate? About a month ago, I ran in some problems because I got a new certificate from RapidSSL which had been signed by their new-ish SHA256 certificate but older clients (~3 years old browsers) had only RapidSSLs old root certificate installed.
    - ezequiel-garzon 12y ago
      Thanks for answering. I'm not sure, but wouldn't this be a widespread issue, given the popularity (read free as in beer) of StartCom certificates? I'm pretty sure I'm doing something wrong.

floatboth 12y ago

Why do they even have an httpd in base? They like to say they're smaller and simpler than FreeBSD, but FreeBSD doesn't include a web server in base!

clarkevans 12y ago
It's very handy to wrap services in HTTP to expose them outside the machine, limiting outside access to a single port with clear FastCGI delegations to specific functions. If you don't build this in, then services shouldn't assume it to be operational and can't auto-configure themselves.
stsp 12y ago
One reason is that network installs require http. The base system would be unable to self-replicate across network installs without httpd.
- floatboth 12y ago
  Why not FTP? Or even TFTP?
  - pyvpx 12y ago
    you can do that as well.
  - tedunangst 12y ago
    FTP is dead.
    - wglb 12y ago
      How I wish this were true.
      - protomyth 12y ago
        Yeah, don't you just love how "Enterprise" vendors make it more secure with going with ftps instead of picking a different protocol?

joosters 12y ago

OpenBSD seems to have caught a bad case of the 'not invented here' sickness. If they didn't like where nginx was going, why not just fork it and have a working web server with a known codebase? The forks would diverge but they could still grab fixes from nginx whenever they wanted to.

clarry 12y ago
See how well forking has worked in the past? Especially with large components like compilers or web servers (see old apache, old gcc). Such forks end up antiquated and more or less stagnant. Besides, grabbing fixes just like that doesn't work so well once you've diverged enough and the upstream is constantly growing and changing. RTFA and you'll see this point discussed.
stsp 12y ago
This wasn't a question about having a functional webserver or not. It was a conscious decision against running code without reading it first. The "run but don't read" strategy belongs into ports, not base. Hence nginx moved to ports. Forking nginx was actually discussed when a proposed nginx update diff was too large for proper review. Tons of complex regex parsing code was added with nobody willing to go through it all in detail. The forking option was quickly dismissed. The tipping point happened when someone (reyk?) pointed out that relayd had most of the guts of a complete web server anyway, including OpenBSD-style privsep which has to be bolted on to virtually all software imported from elsewhere, including nginx. It took about a week from that discussion happening to having a relayd-based functional and peer-reviewed web server in base (+ a couple of months of shaking out a few bugs and adding minor features).
- jimktrains2 12y ago
  > he "run but don't read" strategy belongs into ports, not base. Hence nginx moved to ports. Why is there a webserver in base?
  - jquast 12y ago
    For the same reason the other 20+ net-daemons are: http://cvsweb.openbsd.org/cgi-bin/cvsweb/~checkout~/src/etc/rc.conf?content-type=text/plain http://cvsweb.openbsd.org/cgi-bin/cvsweb/~checkout~/src/etc/... With OpenBSD, you can actually avoid the ports tree all together and still play a critical role in your infrastructure. If you're going to distribute a system that doesn't do much of anything on its own, and delegate the responsibility of security (and choice!) to "the ports tree" you're not doing any better than Linux or FreeBSD.

cturner 12y ago

What would be the elegant way to implement websockets on the new openbsd arrangement? Would it be to use relayd instead of httpd? Or is websocketd suitable for the openbsd base?

detaro 12y ago
Probably to install nginx from the ports and use that. I don't think supporting web sockets would fit the scope of the base system.

zx2c4 12y ago

I love these guys: https://github.com/reyk/httpd/blob/master/httpd/server_http.c#L802-L804 https://github.com/reyk/httpd/blob/master/httpd/server_http....

fredsted 12y ago
Hmm, Comic Sans, weird choice for an error message.
fafner 12y ago
At least they added Comic Neue. I'm a bit disappointed that a project like OpenBSD that is so vocal about free software is promoting a proprietary font like Comic Sans MS.
- termie 12y ago
  Not sure if whooshed, but just in case anyone else doesn't get it, you're not going to see Comic Sans unless you load it from a Windows machine or a Unix distribution that's installed the font. IOW, it's just a gentle jab at Microsoft and perhaps those distributions that install the MS fonts. Technically that is promotion in name but not spirit.
  - fafner 12y ago
    Well the OpenBSD devs seem to use Comic Sans in their presentations though E.g., http://www.openbsd.org/papers/ru13-deraadt/index.html http://www.openbsd.org/papers/ru13-deraadt/index.html
    - termie 12y ago
      You're right, they use it as a weapon of snark and for fund raising: http://www.openbsd.org/papers/bsdcan14-libressl/mgp00025.html http://www.openbsd.org/papers/bsdcan14-libressl/mgp00025.htm...
  - setpatchaddress 12y ago
    You'll see it on the Mac, too, since the MS web fonts are installed by default.

jalfresi 12y ago

Does anyone know if the FastCGI implementation is complete i.e. it supports FastCGI processes in all three roles; Responder, Authorizer and Filter? I've always wanted to use FastCGI more but most implementations (in Apache and Nginx at least) only support some of those roles (or require work arounds using server specific features; i.e. apache filters rather than FastCGI filters)

tedunangst 12y ago
I think it's "it runs the software we tested with" complete.

mdekkers 12y ago

any performance benchmarks in the wild?

jnazario 12y ago

[update - i read the back story elsewhere and the reason is less boneheaded than i had assumed. still, i think the community needs to focus on higher priority needs and gaps] this is the sort of thing that makes me happy i'm no longer involved in the OpenBSD world. httpd & previously smtpd are two replacements that (in my opinion) have little additive value beyond existing, community-adopted solutions (e.g. nginx and postfix), diluting effort where it is needed. does the world need a new httpd? maybe. but the world needs other replacement software to be done first because it'll have a greater impact. for example, OpenBSD could invest time and effort in maturing static code analyzers to assist in code audits (especially of ports). i suspect this new httpd was done less because it was needed and more because it could be done. that's the attitude i disagree with.

tedunangst 12y ago
Not all developers are fungible.
- deleted 12y ago
  [deleted]
p9idf 12y ago
If the world needed OpenBSD programmers to write high-impact replacement software so badly, the world would offer to pay for it. And since evidently no one wants to pay them to do so, it seems pretty reasonable to expect that the unpaid programmers -- volunteering their free time to write programs that they give away to the world for free -- would set their own priorities, address their own needs, and invest their time and effort however they see fit.
floatboth 12y ago
OpenSMTPD is much, much simpler than Postfix. I can't stand Postfix's fucking 1000 line multi-file configuration. Nginx is simple, so, yeah – no need for a replacement there.
- ta0967 12y ago

marc_omorain 12y ago

Is there a technical reason why you would implement HTTPS in a HTTP server? If you ran a separate process on port 443 to terminate SSL connections, and then proxy that request to a HTTP server running locally, there would be better separation of concerns. For example, this setup would mean that a security flaw in the HTTP server that allowed a user to read memory would not be able to read any private keys used in the HTTPS server. I guess some downsides would be some extra latency while the request is proxied, and some extra memory overhead for the second process. I'm interested in anyones thoughts on this.

Too 12y ago
I'm not into the specifics of https but i've been writing a lot of gateways for other protocols and it's never as easy as "just forward the message". Protocols some times have state that the proxy must be aware of and sometimes the forwarding is conditional which means that the proxy must understand both types of protocol and be able to act based on information in it. Say for example you want to block all requests to a specific resource, if your https server knows about this you might be able to reject the request before you decrypt everything of it.
throwaway2048 12y ago
OpenBSD has added support into libressl for privilege separated processes that hold SSL keys, any operation requiring the use of private keys such as the creation of session keys, and signing things are shuttled off via a small api to a separate process. This is somewhat analogous to what ssh-agent does for openssh clients. OpenBSD's TLS private key consuming daemons have moved to this model or are in the process of doing so. This helps to mitigate the problem of access to process memory results in disclosed private keys, also the requirement of the daemon's user facing bits to have access to the keyfiles. http://article.gmane.org/gmane.os.openbsd.cvs/139527/ http://article.gmane.org/gmane.os.openbsd.cvs/139527/
zurn 12y ago
Proxying typically loses a lot of information, like client address, client certificate info and so on. If you take care to make carry over everything transparently, it would more accurately be called privilege separation (ala sshd).

amelius 12y ago

I'm amazed we still use configuration files like that. Why not have some capable programming language (python, javascript) handle the configuration? This would let one more naturally describe things, and will eventually reduce the possibility of obscure configuration commands that do one very specific thing, and hence lead to frustration when the user needs something slightly different. Using a programming language, one could even configure using callbacks, etc. and is much more flexible.

m0th87 12y ago
I'm not a lisper, but this makes me pine for some universal, maybe lisp-like language. It's a total waste of time to learn a new config syntax for each application.
- gjggkjasdf 12y ago
  the ironic thing here is that Lispers fucking love sticking new DSLs wherever they possibly can. And I imagine most developers of these pet config files would argue the same thing. Unfortunately. It was Microsoft of the 1990s that surprisingly brought some sanity to the table, with INI files. I'm glad tools like Git use this format.
  - ams6110 12y ago
    Then they threw sanity out the window with the registry.
- throwaway2048 12y ago
  This is getting pretty offtopic, but this is what GNU has tried to do with guile https://en.wikipedia.org/wiki/GNU_Guile https://en.wikipedia.org/wiki/GNU_Guile
- catwell 12y ago
  Lua is often used for that purpose. It is not a Lisp but it fulfills this role well enough.
12y ago

davidgerard 12y ago

I read the config file format and I fell in love. I really hope this gets the portable treatment.

davidgerard 12y ago

I read the config file format and I fell in love. I really hope this gets the portable treatment.

captaintod 12y ago

Surprised that nobody has mentioned ashd yet: http://www.dolda2000.com/~fredrik/ashd/ http://www.dolda2000.com/~fredrik/ashd/

dyoder 12y ago

Did OpenBSD just standardize on an HTTP server they wrote in 2 weeks, has no tests, doesn't fully implement the spec…and then brag about it?

tedunangst 12y ago
No.

kymywho 12y ago

Https authentication support for Subversion could be the killer feature.