14 ms·

Announcing Notqmail

160 points by cpeterso 7y ago

nineteen999 7y ago
I have to wonder how much work is going to be required to bring qmail up to a modern standard. There's no doubt that the author is an extremely intelligent programmer with a deep focus on security. In the intervening period Postfix has really gained in popularity, and is shipped pre-configured out of the box on some major Linux distributions. It is the default MTA for Fedora/Redhat etc for some years now. I wanted to like qmail, it was definitely an improvement over the complicated disaster that is Sendmail, however, it always seemed to require patches to get it to do things that Postfix could already do out of the box. In addition I didn't like the dependency on the inetd replacement thing (daemontools?). It just felt so un-ergonomic. In addition, the Maildir mailbox format (while very cool in its own right) usually required third-party software to be patched manually, for example WU-IMAP. A day or two before the September 11 attacks, the UNIX time_t value rolled over to 1 billion, and went from 9 digits long to 10 digits long. The moment this happened, a comparison function in the WU-IMAP maildir patches failed, and our IMAP server started sorting and displaying all our emails out of order. After scratching our heads and poring over the source code for a couple of hours, a co-worker and I found the problem and submitted a patch to the maintainer. Funnily enough, the imap-maildir page still references this event even today (it probably hasn't been updated in years): http://www.davideous.com/imap-maildir/ http://www.davideous.com/imap-maildir/ Under the section "10-digit unix date rollover problem". But it drive home to me as a fairly young sysadmin at the time of the dangers (at least for the inexperienced) of using third-party patches just in order to enable a non-standard mailbox format (eg. Maildir).
- codetrotter 7y ago
  From the page you linked: > Did your installation stop sorting messages correctly on 9 Sep 2001 01:46:40 GMT? Find the fix to the 10-digit unix date rollover problem in the important updates section below. Reading that I imagined someone running those patches on their mail server for two decades before suddenly realizing one day in 2019 that for the past 18 years their messages have been ordered incorrectly, ha ha.
- schmonz 7y ago
  It will be a lot of work, no question about that. Fortunately none of it will be about Maildir, as folks like you have long since ironed out all those issues (thanks!). It's possible to package qmail in such a way that it's trivial to install and run, supporting many modern features by default. I've done it. Here's a demo: https://youtu.be/Vq6vu9T3vow https://youtu.be/Vq6vu9T3vow But that required a lot of decision-making and a lot of effort by the packager. With notqmail, we hope to make packaging much much easier.
  - nineteen999 7y ago
    I wish you the best of luck with the project, I hope you are able to achieve all your goals.
    - schmonz 7y ago
      Thank you!
dsr_ 7y ago
I used qmail for about ten years. It was remarkably good from a sysadmin's perspective: - the config system mostly used the name of a directory where lots of software would use a filename (qmail.ini => /var/qmail/conf), the name of a file where other software would use a variable name, and the contents of the file as the value of the variable. This is really, really easy to read and write without writing a parsing library. - separate executables do separate jobs, with well-documented interfaces between them. Oh, and if a part didn't need root, it didn't run as root, and if a part didn't need to be the same user as another, it didn't share users. While it was easiest to replace something at the edge of the graph, it was entirely possible to replace middle jobs as well. - qmail was the original design home of Maildir. I approve of various Maildir++ implementations (nesting folders is useful) and you could add support for them by replacing your delivery agent. - qmail expected to be controlled by daemontools, a supervision system that has inspired/been blatantly copied and improved upon as runit, perp, s6, and nosh. (I have a suspicion that nosh is what most critics of systemd is really looking for.) - fast. - nearly bug-free. What qmail gave up in exchange for all this: modern features and required antifeatures... which, it appears, notqmail is going to implement. Huzzah!
- sneak 7y ago
  > the config system mostly used the name of a directory where lots of software would use a filename (qmail.ini => /var/qmail/conf), the name of a file where other software would use a variable name, and the contents of the file as the value of the variable. This is really, really easy to read and write without writing a parsing library. this djb-ism has seriously affected a lot of the ways i unix, to date. envdir is such a great idea.
  - nailer 7y ago
    djb needs his own distro. He's written his own init, has thoughts about package management, config management, etc. He probably doesn't need his own kernel but a BSD or Linux kernel with DJB controlling userland would be interesting.
- linsomniac 7y ago
  I liked a lot of things about qmail, but I found it annoying as a sysadmin, like in the 1.07 days, because you had to do all this patching. I eventually wrote a script that would take all the sources and patches that one needed to run a reasonable mail server, and turn them into an RPM that could be installed, because I couldn't just provide the patched, resulting RPMs. There were a lot of great things about qmail. I really liked the envdir and how secure it was. But eventually I found the attitude behind it very tiring and switched to Postfix, which is a really great program.
Nux 7y ago
Used qmail a lot and would use it again if modernised a bit. Vpopmail was great, too for managing users, hope it'll still work.
bifrost 7y ago
I haven't used Qmail since 1999 maybe, I don't miss it at all but its kinda nice that some people have modernized DJB's code.
cryptonector 7y ago
Why not Postfix? I trust Wietse V. and Viktor D. a great deal, and I trust their coding abilities at least as much as if not more than DJB's for anything that isn't a cryptographic algorithm. FYI, "I’ve learned more C, reduced build-time complexity, ..." probably isn't confidence-inspiring. I work with C a great deal and have for a very long time, and I though I'm very comfortable with C, I treat it with fear and respect -- I have found far too many security vulnerabilities just from staring at C source code to do anything else. Now, I too have "learned more C" over the years, but I wouldn't write a sentence like that as part of justification for some project written in C -- I would fear that others would wonder if I don't just know enough C to be dangerous. Perhaps one should not write "I've learned more C" as part of anything other than justification for moving to Rust.
- stonogo 7y ago
  Postfix cannot do what qmail can do. Specifically, users can create and destroy email addresses themselves with .qmail files. This is different and more powerful than Google's plus-addressing approach. There are many, many other features of qmail's implementation that Postfix cannot compete with. Postfix is not alone in this; no other MTA/MDA can do what qmail can do.
  - pmoriarty 7y ago
    It would be really useful to have an exhaustive list of all these things that qmail can do that other MTA's can't.
    - pilif 7y ago
      it would be a very small list. However, the exhaustive list of things other MTAs can do that qmail can't would be huge (starting with backscatter-proof bounces for example) qmail was limited back in the 90ies and it's horribly behind the times now.
  - amaccuish 7y ago
    I've got a similar thing with my postfix setup though. My users can go to their account portal and apply for an alias, which when approved (to avoid taking of important names like postmaster etc.) is added to their otherMailboxes LDAP attribute. They can delete the alias themselves when they're done with it. I'm tempted to start auto-allowing new aliases and just having a list of names that can't be used (postmaster, webmaster) etc, that some services use for domain ownership verification. As an aside I do hope qmail offers similar limiting abilities. It wouldn't be great if a user could make themselves receive webmaster@domain.tld emails!
lwh 7y ago
We all killed SMTP when we pointed the morons with domains at gmail/apple/yahoo/o365, postfix was winning when it was ended- no reason to revisit it. All that matters for email is what Google/MS/Apple decide to dictate to us now.
intc 7y ago
Just wanted to add that Erwin Hoffmann has actively maintained s/qmail (and related djb tools) for quite some time. =) Please feel free to check his website: * http://www.fehcom.de/sqmail/sqmail.html http://www.fehcom.de/sqmail/sqmail.html
- schmonz 7y ago
  Yes, and while I haven't run s/qmail myself, I have sometimes borrowed (and then usually refactored) some of his code, and have frequently collaborated with him on improving ucspi-tcp6, ucspi-ssl, etc. I'm still hoping he'll consider participating in notqmail development.
genghizkhan 7y ago
I run my own email server as well, and I've been using postfix/dovecot quite happily with zero issues. I've been contemplating opensmtpd as well, though I'm not sure where that stands when it comes to being run in production environments. Also, reconfiguring mailservers is a PITA, I'd rather stick with a setup which just works. Could someone explain why one might want to use qmail (or this, its newest incarnation) over postfix or opensmtpd? Would I gain anything by shifting to qmail or opensmtpd?
- schmonz 7y ago
  For the moment, we're targeting sysadmins who already run qmail (or netqmail). Much later, when we begin to approach feature parity with Postfix, notqmail might start being an interesting alternative. We hope that time will come. If you can't figure out why you'd run it right now, you're probably right :-)
  - genghizkhan 7y ago
    Thank you! I will definitely keep a weather eye on this project! :)
- poolpOrg 7y ago
  I'm an opensmtpd developer. As far as opensmtpd is concerned, it is production ready and has been used in high volume environments sending mails to millions of recipients daily, handling multi-million queues. The project was started in 2008 so when it comes to being run in production environments, I think we can claim that it's ready :-) Now as for the what you would gain, I won't do proselytism, we have published a couple presentations on our website which describe how it works and some of the technical details behind it. You will also find lots of technical details on my blog where I post mostly about OpenSMTPD internals. If you have a running Postfix, I suggest you don't shift but keep your working setup. If you intend to setup a new MX, you should definitely give a try and see how it goes for you.
  - genghizkhan 7y ago
    Thank you very much! I shall have a look. I'm thinking of moving servers and setting everything up from scratch in order to modernize all my services and remove legacy cruft. I will definitely consider opensmtpd.

160 points by cpeterso 7y ago

I have to wonder how much work is going to be required to bring qmail up to a modern standard. There's no doubt that the author is an extremely intelligent programmer with a deep focus on security. In the intervening period Postfix has really gained in popularity, and is shipped pre-configured out of the box on some major Linux distributions. It is the default MTA for Fedora/Redhat etc for some years now. I wanted to like qmail, it was definitely an improvement over the complicated disaster that is Sendmail, however, it always seemed to require patches to get it to do things that Postfix could already do out of the box. In addition I didn't like the dependency on the inetd replacement thing (daemontools?). It just felt so un-ergonomic. In addition, the Maildir mailbox format (while very cool in its own right) usually required third-party software to be patched manually, for example WU-IMAP. A day or two before the September 11 attacks, the UNIX time_t value rolled over to 1 billion, and went from 9 digits long to 10 digits long. The moment this happened, a comparison function in the WU-IMAP maildir patches failed, and our IMAP server started sorting and displaying all our emails out of order. After scratching our heads and poring over the source code for a couple of hours, a co-worker and I found the problem and submitted a patch to the maintainer. Funnily enough, the imap-maildir page still references this event even today (it probably hasn't been updated in years): http://www.davideous.com/imap-maildir/ http://www.davideous.com/imap-maildir/ Under the section "10-digit unix date rollover problem". But it drive home to me as a fairly young sysadmin at the time of the dangers (at least for the inexperienced) of using third-party patches just in order to enable a non-standard mailbox format (eg. Maildir).

codetrotter 7y ago
From the page you linked: > Did your installation stop sorting messages correctly on 9 Sep 2001 01:46:40 GMT? Find the fix to the 10-digit unix date rollover problem in the important updates section below. Reading that I imagined someone running those patches on their mail server for two decades before suddenly realizing one day in 2019 that for the past 18 years their messages have been ordered incorrectly, ha ha.
schmonz 7y ago
It will be a lot of work, no question about that. Fortunately none of it will be about Maildir, as folks like you have long since ironed out all those issues (thanks!). It's possible to package qmail in such a way that it's trivial to install and run, supporting many modern features by default. I've done it. Here's a demo: https://youtu.be/Vq6vu9T3vow https://youtu.be/Vq6vu9T3vow But that required a lot of decision-making and a lot of effort by the packager. With notqmail, we hope to make packaging much much easier.
- nineteen999 7y ago
  I wish you the best of luck with the project, I hope you are able to achieve all your goals.
  - schmonz 7y ago
    Thank you!

dsr_ 7y ago

I used qmail for about ten years. It was remarkably good from a sysadmin's perspective: - the config system mostly used the name of a directory where lots of software would use a filename (qmail.ini => /var/qmail/conf), the name of a file where other software would use a variable name, and the contents of the file as the value of the variable. This is really, really easy to read and write without writing a parsing library. - separate executables do separate jobs, with well-documented interfaces between them. Oh, and if a part didn't need root, it didn't run as root, and if a part didn't need to be the same user as another, it didn't share users. While it was easiest to replace something at the edge of the graph, it was entirely possible to replace middle jobs as well. - qmail was the original design home of Maildir. I approve of various Maildir++ implementations (nesting folders is useful) and you could add support for them by replacing your delivery agent. - qmail expected to be controlled by daemontools, a supervision system that has inspired/been blatantly copied and improved upon as runit, perp, s6, and nosh. (I have a suspicion that nosh is what most critics of systemd is really looking for.) - fast. - nearly bug-free. What qmail gave up in exchange for all this: modern features and required antifeatures... which, it appears, notqmail is going to implement. Huzzah!

sneak 7y ago
> the config system mostly used the name of a directory where lots of software would use a filename (qmail.ini => /var/qmail/conf), the name of a file where other software would use a variable name, and the contents of the file as the value of the variable. This is really, really easy to read and write without writing a parsing library. this djb-ism has seriously affected a lot of the ways i unix, to date. envdir is such a great idea.
- nailer 7y ago
  djb needs his own distro. He's written his own init, has thoughts about package management, config management, etc. He probably doesn't need his own kernel but a BSD or Linux kernel with DJB controlling userland would be interesting.
linsomniac 7y ago
I liked a lot of things about qmail, but I found it annoying as a sysadmin, like in the 1.07 days, because you had to do all this patching. I eventually wrote a script that would take all the sources and patches that one needed to run a reasonable mail server, and turn them into an RPM that could be installed, because I couldn't just provide the patched, resulting RPMs. There were a lot of great things about qmail. I really liked the envdir and how secure it was. But eventually I found the attitude behind it very tiring and switched to Postfix, which is a really great program.

Nux 7y ago

Used qmail a lot and would use it again if modernised a bit. Vpopmail was great, too for managing users, hope it'll still work.

bifrost 7y ago

I haven't used Qmail since 1999 maybe, I don't miss it at all but its kinda nice that some people have modernized DJB's code.

cryptonector 7y ago

Why not Postfix? I trust Wietse V. and Viktor D. a great deal, and I trust their coding abilities at least as much as if not more than DJB's for anything that isn't a cryptographic algorithm. FYI, "I’ve learned more C, reduced build-time complexity, ..." probably isn't confidence-inspiring. I work with C a great deal and have for a very long time, and I though I'm very comfortable with C, I treat it with fear and respect -- I have found far too many security vulnerabilities just from staring at C source code to do anything else. Now, I too have "learned more C" over the years, but I wouldn't write a sentence like that as part of justification for some project written in C -- I would fear that others would wonder if I don't just know enough C to be dangerous. Perhaps one should not write "I've learned more C" as part of anything other than justification for moving to Rust.

stonogo 7y ago
Postfix cannot do what qmail can do. Specifically, users can create and destroy email addresses themselves with .qmail files. This is different and more powerful than Google's plus-addressing approach. There are many, many other features of qmail's implementation that Postfix cannot compete with. Postfix is not alone in this; no other MTA/MDA can do what qmail can do.
- pmoriarty 7y ago
  It would be really useful to have an exhaustive list of all these things that qmail can do that other MTA's can't.
  - pilif 7y ago
    it would be a very small list. However, the exhaustive list of things other MTAs can do that qmail can't would be huge (starting with backscatter-proof bounces for example) qmail was limited back in the 90ies and it's horribly behind the times now.
- amaccuish 7y ago
  I've got a similar thing with my postfix setup though. My users can go to their account portal and apply for an alias, which when approved (to avoid taking of important names like postmaster etc.) is added to their otherMailboxes LDAP attribute. They can delete the alias themselves when they're done with it. I'm tempted to start auto-allowing new aliases and just having a list of names that can't be used (postmaster, webmaster) etc, that some services use for domain ownership verification. As an aside I do hope qmail offers similar limiting abilities. It wouldn't be great if a user could make themselves receive webmaster@domain.tld emails!

lwh 7y ago

We all killed SMTP when we pointed the morons with domains at gmail/apple/yahoo/o365, postfix was winning when it was ended- no reason to revisit it. All that matters for email is what Google/MS/Apple decide to dictate to us now.

intc 7y ago

Just wanted to add that Erwin Hoffmann has actively maintained s/qmail (and related djb tools) for quite some time. =) Please feel free to check his website: * http://www.fehcom.de/sqmail/sqmail.html http://www.fehcom.de/sqmail/sqmail.html

schmonz 7y ago
Yes, and while I haven't run s/qmail myself, I have sometimes borrowed (and then usually refactored) some of his code, and have frequently collaborated with him on improving ucspi-tcp6, ucspi-ssl, etc. I'm still hoping he'll consider participating in notqmail development.

genghizkhan 7y ago

I run my own email server as well, and I've been using postfix/dovecot quite happily with zero issues. I've been contemplating opensmtpd as well, though I'm not sure where that stands when it comes to being run in production environments. Also, reconfiguring mailservers is a PITA, I'd rather stick with a setup which just works. Could someone explain why one might want to use qmail (or this, its newest incarnation) over postfix or opensmtpd? Would I gain anything by shifting to qmail or opensmtpd?

schmonz 7y ago
For the moment, we're targeting sysadmins who already run qmail (or netqmail). Much later, when we begin to approach feature parity with Postfix, notqmail might start being an interesting alternative. We hope that time will come. If you can't figure out why you'd run it right now, you're probably right :-)
- genghizkhan 7y ago
  Thank you! I will definitely keep a weather eye on this project! :)
poolpOrg 7y ago
I'm an opensmtpd developer. As far as opensmtpd is concerned, it is production ready and has been used in high volume environments sending mails to millions of recipients daily, handling multi-million queues. The project was started in 2008 so when it comes to being run in production environments, I think we can claim that it's ready :-) Now as for the what you would gain, I won't do proselytism, we have published a couple presentations on our website which describe how it works and some of the technical details behind it. You will also find lots of technical details on my blog where I post mostly about OpenSMTPD internals. If you have a running Postfix, I suggest you don't shift but keep your working setup. If you intend to setup a new MX, you should definitely give a try and see how it goes for you.
- genghizkhan 7y ago
  Thank you very much! I shall have a look. I'm thinking of moving servers and setting everything up from scratch in order to modernize all my services and remove legacy cruft. I will definitely consider opensmtpd.