14 ms·

How to set up a mail server on a GNU / Linux system

163 points by nsomaru 13y ago

s7an4o 13y ago
This is a very nice article!
kennu 13y ago
That's nice. I have for a long time been running just Postfix + Dovecot IMAP and relying on OSX Mail's spam detection, which is not that great (lots of false positives, learned data not synched between computers, etc). This inspires me to try using a SpamAssassin + ClamAV combo on the server.
captn3m0 13y ago
The ridiculous length of this guide is a testament to the fact that it is still really hard to host your own mail.
- VLM 13y ago
  Something as small as "your own mail" is one or two packages on one server; this guy is implementing most of an entire IT infrastructure including database server and virus scanning capable of operating a hosting company. Sort of like the existence of Oracle's products and IBM's DB2 does not preclude end users being able to use SQLite.
  - trurl42 13y ago
    Virus scanning is maybe not that important, but pretty much everything else is. Spam filtering for example. Would you really want to run a mail setup without it? What would be the "SQLite" in mail servers, including spam filtering, webmail, imap and all the other conveniences?
    - calpaterson 13y ago
      I was able to run my personal mailserver for nearly two years without having to learn what amavis-new was. After that, I started getting enough spam that it became an annoyance and I added it. Webmail is not that important to me now that I have a smartphone (I really recommend K-9 Mail for Android, which is FOSS https://play.google.com/store/apps/details?id=com.fsck.k9&hl=en_GB https://play.google.com/store/apps/details?id=com.fsck.k9&hl...). Failing that can always log into my machine using ssh and use mutt to read and send mail. I think the mailserver SQLite is Postfix (SMTP), Dovecot (IMAP) and some DNS records.
      - trurl42 13y ago
        Postfix is the mail server used in this very guide. So I don't consider Postfix to be easy to set up. OpenSMTPD looks like a step in the right direction.
        calpaterson 13y ago
        I haven't looked at OpenSMTPD. This guide does include a lot of configuration, but it's worth mentioning that not only are the defaults pretty reasonable, but that Ubuntu (and Debian) can configure the most important options using dpkg-reconfigure.
        spudlyo 13y ago
        If you think Postfix is hard to set up, you've probably never had to deal with Sendmail.
    - VLM 13y ago
      SQLite doesn't have all the conveniences of Oracle products or DB2 so that is not a good analogy. I'd define a minimal working system as the distro's MTA set up to BE a smarthost instead of set up to USE a smarthost which is usually not much, and an imap server so "normal" non-mutt using endusers can access it. Everything else is bolt on fun once that basic setup works. Client side spam filtering gets you out of being the middleman, which is awesome if you'd be stuck as the middleman. There's nothing wrong with the article as a display of nearly the largest most complicated system you can design, which is perfectly cool and fun. Personally I'd add commentary to the article about load balancing and proxying and DNS needs more than "yup gonna have to do DNS" but it is very near a maxed out design. Many HN comments are along the lines of "this maxed out design is large and complicated which implies ..." However That doesn't imply anything about non-maxed out designs. The level of what can be accomplished in (insert programming language) in a maximally complicated situation doesn't necessarily imply much about the difficulty of "hello world" Exim and a simple dovecot imap are about ten lines of config editing total. It won't do much, but sometimes that exactly what is needed.
    - gaadd33 13y ago
      I run a mail server with no spam filtering and its not too bad, maybe 2-3 spam messages a day and Thunderbird catches those. I just have a somewhat strict postfix configuration with graylisting enabled. My email is only ~13 years old so I'm not sure how bad it is for people with older domains/emails.
      - clarry 13y ago
        Spam can be hard to predict. I know people who have had a setup like that (working just fine as for you), and then all of a sudden they've been overwhelmed with hundreds to thousands of spam mails per day.
- Sssnake 13y ago
  It really isn't. It is a testament to how many people don't want to learn anything, and just want to follow a list of directions. Setting up your own mail server is trivial, it is a 5 minute job.
- clarry 13y ago
  A mail setup does not have to be this complex. Mine, for example, consists of seven lines of configuration for the daemon, one line to have the init system start the server at boot, plus a few aliases & virtual users. This is using the mail server that comes pre-installed with my OS. It took a few minutes to set up and test. And it's been running for a few years now with no changes.
darklajid 13y ago
This looks like _a lot_ of work was put into the article. Nevertheless, I think you shouldn't do this. This takes a lot of time and is basically copy/pasting things around. For something as crucial as your own mail server I'd go with a solution that is automated: If your server goes away you fire up a different one in a couple of minutes, restore backups and change DNS records, done. Reasons for this might be, in no particular order: - VPS provider goes away - hardware failure - user error (think "Oh, I probably shouldn't have issued that command as root on my mailserver") - maintenance: This guide is (initially) from 2009. How are you making sure that your system is up to date, after you followed the instructions/installed it? Either you'll leave the system to bitrot or you need to actively maintain it and probably will run into issues upgrading the machine. Or .. you migrate to a brand new machine every 6-12 month, which gives you a decent excuse to verify that your backup/restore process is working as well.
- nsomaru 13y ago
  Considering that this can be installed to a VM, an image of which is backed up, what is not automated about this solution? Maintenance: can't the distro package manager take care of that for me? Sorry for the ignorance in my stance, but I came across this article because I'm looking to move away from Google infrastructure to hosting my own stuff, and this seemed like a great piece :)
  - Goladus 13y ago
    VM Images not very portable, and have essentially no built-in documentation. They're also large if you're not working from a snapshot system. Basically, if the exactly prescribed restore procedure fails, a sysadmin is stuck with a virtual black box of bytes and will have to divine out what sort of configuration mismatch there might be between the image and the new host. Different block devices? Different network devices? Wrong CPU architecture? Wrong virtualdisk format? Some other magic of some sort? It can be awhile before you even get the machine up and running, to say nothing about running services. In contrast, a scripted install starts with a running, networked machine and if any point in the install procedure fails, you'll know what point it failed will have a detailed error message explaining why. You can read the script to see what it's trying to do and may be able to either correct the script for the new environment or correct the environment for the script. You can make steady progress towards restoration while if the VM restoration fails, you just keep slamming your head into the brick wall until it gives way. It's true in many situations that restoring a virtual machine snapshot is much faster and easier than a rebuild and data-restore. My experience is that it's great when it works, but a nightmare when it doesn't.
300 13y ago
Respect for all the work done - this is a long but detailed guide!
jvehent 13y ago
I wrote something similar, that uses Postfix, Dovecot, OpenLDAP, Roundcube, DSPAM and OpenDKIM. http://wiki.linuxwall.info/doku.php/en:ressources:dossiers:nectux http://wiki.linuxwall.info/doku.php/en:ressources:dossiers:n... I presented it at my local LUG a while back, here are the slides http://www.slideshare.net/jvehent/di-ymail http://www.slideshare.net/jvehent/di-ymail
ivan_ah 13y ago
Does anyone have experience running a mail server on a home connection with a dynamic IP? I'm imagining a system that will have a homebox (long term storage, privacy) and a cloudbox (provides availability) and the mail flow will be like this: sender ---(1)---> cloudbox ---(2)---> homebox Assuming cloud stays up (1) will happen, and if homebox is reachable, then (2) will also happen. However, if (2) can't be done, then cloudbox will temporarily hold the email until homebox comes online (POP mail style). Has anyone ever setup something like this before? Any pointers will be appreciated.
- e12e 13y ago
  I've not tried the above -- but there shouldn't be any problem with it (although if you control both servers, you might as well not use smtp from 1 to 2). If you want to do this, you could also just try and keep your dns ttl reasonably low, and updated, and recieve stmp straight at home -- after all: "The sender MUST delay retrying a particular destination after one attempt has failed. In general, the retry interval SHOULD be at least 30 minutes; however, more sophisticated and variable strategies will be beneficial when the SMTP client can determine the reason for non-delivery. Retries continue until the message is transmitted or the sender gives up; the give-up time generally needs to be at least 4-5 days." (My emphasis) http://www.ietf.org/rfc/rfc2821.txt http://www.ietf.org/rfc/rfc2821.txt
- VLM 13y ago
  This is pretty well trodden ground, between people here and google of decades of history you'll have no problem. Don't forget to implement the arrows flipped around, I sent all my outgoing thru my cloud server after authentication, so everything goes in and out the same way. Eventually I gave up because in my situation forward everything to gmail is not an issue for me; email is just a spam and amazon receipt delivery service, so its not worth much effort. Anyway one interesting design aspect is you're going to fetchmail from home to a cloud pop server and you'll want to secure that, so the natural inclination is to take smallest possible steps while testing all the way, first you try an unusual port number to keep scanners away, maybe you only allow the big netblocks from your provider (not specific /32 addrs but if your provider gives you an addrs from a /19 well then permit the whole /19) then all manner of SSL/TLS protocols instead of plain text passwords and things like that. But I'd advise saving a lot of time and going big for security before even playing with the mail system. So put openVPN on the cloud provider and have your home and your phones/tablets/laptops/friends/family can VPN into your big happy psuedo-internal LAN... don't let "the internet" connect to your cloud pop (imap?) server at all, only allow connections from internal (via VPN) addresses. You're probably going to end up doing it sooner or later and a VPN'd config obsoletes quite a bit of internet accessible stuff for the pop server. You can have a perfectly static internal LAN addrs for homebox and cloudbox over the VPN, even if endpoint addrs change, which makes some simple ip address firewall rules easier than if both endpoints were dynamic. So by the magic of openvpn 10.1.0.0/16 is in the cloudbox and 10.2.0.0/16 is at home, all the time forever, regardless of current provider both at home or cloud, which makes it pretty easy to create static iptables rules and such. I find it handy to have my phone and tablet always on my home network, because I have fun stuff on my home network. So I'm going to be doing this anyway... may as well use the same security infrastructure to secure my fetchmail sessions rather than trying to secure them on the open internet.
blakesterz 13y ago
I've been doing web/mail hosting since 2002 and it amazes me that it is STILL this hard to do a mail server. I gave up and started using iredmail 2 years ago, so far it's been great.
naterator 13y ago
https://github.com/al3x/sovereign https://github.com/al3x/sovereign
- csense 13y ago
  I've been looking for a good introduction to Ansible -- this may be it!
adders 13y ago
I would definitely recommend using Virtualmin, its a hosting control panel that can control mail/websites/dns in one via a simple control panel. All works like Plesk, but without breaking everything. They've got a script that sets up dovecot, postfix, apache, bind, clamav, spamassassin, Mysql, and loads of other hosting software and then you configure every thing via a web interface. Linux install script is at http://software.virtualmin.com/gpl/scripts/install.sh http://software.virtualmin.com/gpl/scripts/install.sh, instructions at http://virtualmin.com/download http://virtualmin.com/download
plg 13y ago
What do HNers think of the Mac OSX server solution? Assuming you have an apple machine, isn't it rather easy to set up a mail server?
- protomyth 13y ago
  Yeah, last I looked its dovecot and postfix for OS X. It is a pretty easy setup. Just buy server from the App Store and follow the instructions.
N0RMAN 13y ago
Is it recommended to use a relay for postfix (for private usage) like Mailgun?
spindritf 13y ago
I run my own e-mail server and it's a lot of work. I have exim with ACLs, spamd and just mutt. Nothing "fancy" like other users, pop3/imap/smtp auth, virus scanning, additional mxs... Still took me a weekend to get ACLs right, tune the threshold for greylisting (I greylist e-mails which score between two and seven points in Spam Assasin), set up DKIM and SPF... it's not difficult but there's a long list of things to do. If you're doing it just for yourself, exim, spamd and mutt are enough. But I wouldn't want to be responsible for a larger installation with regular, non-technical users. It's not just the configuration, your service will always be subpar. At least until there's a good webmail (I have high hopes for Mailpile).
- strumptrumpet 13y ago
  I run my own mail server and it's essentially zero work. I have postfix, dovecot, SpamAssassin, and RoundCube, all authenticated via LDAP. It required some upfront configuration, but it has run itself with minimal configuration changes and 'apt-get' updates for almost 6 years now. I never touch it, and it works. > It's not just the configuration, your service will always be subpar. At least until there's a good webmail (I have high hopes for Mailpile). That assumes that user's want to primarily use webmail. In this renaissance of Mac OS X mail clients and mobile phones, that hasn't been the case here.
  - spindritf 13y ago
    LDAP does make things a little easier but how do you synchronize stored e-mails and filtering rules between servers?
    - rhizome 13y ago
      Why do you need to?
    - ibotty 13y ago
      when using dovecot you can use dsync: http://wiki2.dovecot.org/Tools/Dsync http://wiki2.dovecot.org/Tools/Dsync
raimue 13y ago
Good tutorial with detailed set up instructions. On some configuration values I would say the default would suffice as well, but mentioning what can be configured might also be helpful. However, from experience, setting up a database is overkill for a small mail server instance hosting only a few mail boxes. It's way easier to use system user accounts (just disable remote login for them). That avoids the hassles of setting up the virtual mail delivery. Only for a large mail server using a database backend would make sense to gain more performance.
zobzu 13y ago
I run my own email since about 20 years. My current setup is different from the author and I like it better. I had his setup a few years ago I think. - postfix - Dovecot - RoundCube - SpamAssassin w/ sa-learn sa-update and razor - Clamav + Clamav-milter - MariaDB Spam: Not an issue. Features: DKIM, DMARC, SPF checks Maintenance: what maintenance? I update the programs with my distro package manager and that's basically it.. Whoever talked about this being hard.... Webmail: I like RC much better than gmail, nuff said.
- mikevm 13y ago
  What about the reliability of your home Internet connection? Isn't that a major issue? You can have a power outage, ISP problems, or your server might simply die. What happens to the important emails you might miss?
  - nacs 13y ago
    Mail servers are made to attempt redelivery if a mail is undeliverable. Most mail servers will attempt redelivery for multiple days before disposing of it so it shouldn't be an issue.
    - dingaling 13y ago
      Furthermore, to address the 'server on fire' issue. If you build the mail server and its ancillaries up in a VM ( or number of VMs ) then you can sync VM to another host on a schedule, or even live. Set-up a heartbeat ping and the alternative host can bring-up its copy of the mail server VM in a few seconds.
  - zobzu 13y ago
    No big deal, its pretty reliable, but also, but i got an MX at a friend's place. Never lost a mail, even if my server goes down for a couple of days. Mail actually retries for a certain amount of time by default, so short downtime generally causes no problem at all. Longer ones do require the 2nd MX, but yeah, I do that. In about 15 years i think the backup MX helped twice so far (which is probably totally worth it) the mail itself is also replicated there in fact
dschiptsov 13y ago
Few hours of work: sudo apt-get/yum install postfix dovecot openldap roundcube.) OpenLDAP schemes and ssl certs are the bulk of it. Hint: never put postfix's spool and dovecot's storage on the same physical device and I/O controller. /var/log must live on the separate device too.
- brdrak 13y ago
  > never put postfix's spool and dovecot's storage on the same physical device and I/O controller. /var/log must live on the separate device too. This probably depends on the mail volume, no? I host all mail related services for myself and some family on a $9/month VPS (1 VCPU, 512 MB RAM, and pedestrian I/O capabilities) without any issues. My mail volume is typically under 20K messages/day.
borplk 13y ago
The problem with things like this is, since I'm not super confident with any of the technologies involved, it leaves me with an uneasy feeling of having messed something up somewhere. It is extremely easy to set or forget one extra parameter or something and everything falls apart.
ams6110 13y ago
I'd have a look at one of the newer kids on the block, OpenSMTPD. Being from OpenBSD, the man pages are actually good. http://www.opensmtpd.org/manual.html http://www.opensmtpd.org/manual.html
- sliverstorm 13y ago
  Ugh. I want to use OpenBSD- alas, OpenVZ...
bratsche 13y ago
Out of curiosity: people who are running their own mail servers for multiple users, where do you host?
egoitz 13y ago
I would change that Courier or any Dovecot with a Cyrus IMAP implementation... it really rocks in terms of stability (and works smoothly), is fast and replication :)
- houk 13y ago
  Really? My experience has been the complete opposite. Almost every time I've even been involved with or ran any medium/large mail infrastructures (200K active mailboxes) I've always found Dovecot to be a better choice.
  - brdrak 13y ago
    Good to hear that Dovecot scales. I personally went with Cyrus because at the time I was picking an IMAP server, Cyrus was already a well tested solution known to scale. As I recall, Dovecot back then was still young and developed by one person. I've heard a lot of good things about it since though. I'm sticking with Cyrus since it works fine for me, but it's good to have options.
- brdrak 13y ago
  Another satisfied Cyrus IMAP user here. :-) I've heard nice things about Dovecot too, though.
exratione 13y ago
I moved off Abrahamsen's recipe 18 months ago or so for no good reason other than I like to make work for myself: it had been working just fine for me. You set it up on a cloud server, take an image, and that's that - you have a working base from which you can restore your mail server if anything bad should happen. Last year I put up the Dovecot / Postfix / Postfix Admin / Horde recipe I used in place of Abrahamsen's, and judging by the feedback it's helped a great many people: https://www.exratione.com/2012/05/a-mailserver-on-ubuntu-1204-postfix-dovecot-mysql/ https://www.exratione.com/2012/05/a-mailserver-on-ubuntu-120... (You can replace Horde with Roundcube or any other webmail package you care to put in place if you don't like it). The market here is for learning, not necessarily for setting up. If you don't know how to set up a mail server, it is well worth walking through one of these as it will teach the lay of the land. Then you can later move to a packaged solution, as you'll be knowledgeable enough to troubleshoot it when it breaks. I was intending to put together a Chef cookbook for my recipe, but looking at what's already out there it seems like it would duplicate some good ones that already exist. e.g.: https://github.com/onddo/postfix-dovecot-cookbook https://github.com/onddo/postfix-dovecot-cookbook I think that running your own mail server is certainly worth the initial effort; once you have the thing set up it ticks along with very little upkeep needed.
shabbyrobe 13y ago
I have tried this guide and others a couple of times. Unfortunately, no matter what I do, Gmail and Hotmail always classify anything I send from it as spam. DKIM? Check. SPF? Check. Reverse DNS? Check. Spamassassin score? 0.0. Result? "google thinks this message is spam". While it was an interesting experience, the end result is invariably a server that can't send mail to most of the people it needs to, which is quite frustrating.
- Kototama 13y ago
  Here some links that may help you: http://www.brandonchecketts.com/emailtest.php http://www.brandonchecketts.com/emailtest.php http://mxtoolbox.com/ http://mxtoolbox.com/ http://www.codinghorror.com/blog/2010/04/so-youd-like-to-send-some-email-through-code.html http://www.codinghorror.com/blog/2010/04/so-youd-like-to-sen...
jwatte 13y ago
I remember actually understanding sendmail.cf syntax. Those were the days! The problem is not the daemons; those are easy. The problem is spam on the inbound and deliverability on the outbound. And I do /not/ want to fight that fight on my own!
- brdrak 13y ago
  I've personally not found this difficult. Unless perhaps you're dealing very large volumes of mail and overhead of SpamAssassin is an issue. Mail volume in 10s of thousands/day on a low end VPS has been no problem for me.
voltagex_ 13y ago
How do you handle cut-over to your server when waiting for the DNS changes to propagate?
dirkdk 13y ago
Can I ask the obvious question Why? Email (IMAP/SMTP) is so standard, there are lots of companies out there that do it better and cheaper than you yourself can do. GMail (Google Apps for business) and Fastmail.fm are the 2 names I always recommend. Another factor to consider is how you will keep your SMTP ip-addresses from being blacklisted. Companies like Mailchimp and Sendgrid spend a lot of time and effort on this. With your own very low volume, it takes only a couple of persons to mark your email as spam and suddenly all mails sent from your domain end up in in spam folders, no matter what DKIM or SPF settings you have configured. Only reason I would see is when you want to keep your email away from the NSA and therefore out of the USA (Fastmail is Australian based btw http://thenextweb.com/insider/2013/10/07/are-overseas-based-companies-free-from-nsa-requests-australias-fastmail-thinks-so/ http://thenextweb.com/insider/2013/10/07/are-overseas-based-...). Keeping your server out of the USA will not mean that your email will never travel through the USA, where the NSA can still access it, as SMTP is very cleartext between MTA's.
- ivanstojic 13y ago
  At the risk of sounding like another one of the "Google is evil" people, I'd like to voice my concern that Google is better at managing mail then I am. I just moved from one country to another and I can't change the billing settings on my Google Apps account because... Google Apps don't support that. The official stance is that I should just download a backup of my accounts, delete them, recreate them with different billing options. However, other than preserving my mail and Google Drive, this won't preserve any of my other content which must either be backed up using third party tools and then manually recovered/recreated. I'm not sure how this can be interpreted as a sign that Google can do this better than I can.
- Goladus 13y ago
  As much as the NSA concerns me, and as a political issue it's far more important, I am interested in reducing dependency on Google because as far as I am concerned Google has broken its trust with me all by itself. It has nothing to do with the NSA debacle. I don't have time for a rant but the short is: I'm sick of their identity management, their willingness to put blatantly obvious "search bubble" results based on recent (or even non-recent) activity-- stuff that is prominently visible to anyone who might be sharing a screen with me. Ditto for chrome's "most visited pages" thumbnails when you open a new tab, which I disabled but had to install a plugin to do. There's some other minor issues. But the final straw came a few weeks ago when my search history showed up unexpectedly on my girlfriend's phone-- I suppose because she logged into chrome on my laptop to check her Gmail. Until she logged out, auto-complete on her phone was using MY search history from chrome on my desktop. What the FUCK? Obviously I trust her enough to let her log into my laptop so it didn't cause me problems here, but I had no way to know that would happen. So what's it going to be next time? What's google's next clever/reckless silicon valley echo chamber idea that's going to cause me problems? I'd rather not wait to find out. So I've begun the process of reducing google dependence. Email is a big one, Gmail is my single biggest google dependency.
z92 13y ago
save.
locusm 13y ago
Comprehensive, well written article.
robomartin 13y ago
This is really cool. Amazing work. I am looking at running my own mail server out of a Linode instance. It would service a number of our domains and serve to migrate from having web and email live on the same machine at other VPS's. I want to have email serviced from a dedicated instance that does nothing but email for all of our sites. This seems like a better overall strategy but it's been hard to get motivated to get it done due to the amount of work this represents. I have not studied this guide in detail yet but it seems to really take you down the path step by step very nicely. Thanks!

163 points by nsomaru 13y ago

This is a very nice article!

kennu 13y ago

That's nice. I have for a long time been running just Postfix + Dovecot IMAP and relying on OSX Mail's spam detection, which is not that great (lots of false positives, learned data not synched between computers, etc). This inspires me to try using a SpamAssassin + ClamAV combo on the server.

captn3m0 13y ago

The ridiculous length of this guide is a testament to the fact that it is still really hard to host your own mail.

VLM 13y ago
Something as small as "your own mail" is one or two packages on one server; this guy is implementing most of an entire IT infrastructure including database server and virus scanning capable of operating a hosting company. Sort of like the existence of Oracle's products and IBM's DB2 does not preclude end users being able to use SQLite.
- trurl42 13y ago
  Virus scanning is maybe not that important, but pretty much everything else is. Spam filtering for example. Would you really want to run a mail setup without it? What would be the "SQLite" in mail servers, including spam filtering, webmail, imap and all the other conveniences?
  - calpaterson 13y ago
    I was able to run my personal mailserver for nearly two years without having to learn what amavis-new was. After that, I started getting enough spam that it became an annoyance and I added it. Webmail is not that important to me now that I have a smartphone (I really recommend K-9 Mail for Android, which is FOSS https://play.google.com/store/apps/details?id=com.fsck.k9&hl=en_GB https://play.google.com/store/apps/details?id=com.fsck.k9&hl...). Failing that can always log into my machine using ssh and use mutt to read and send mail. I think the mailserver SQLite is Postfix (SMTP), Dovecot (IMAP) and some DNS records.
    - trurl42 13y ago
      Postfix is the mail server used in this very guide. So I don't consider Postfix to be easy to set up. OpenSMTPD looks like a step in the right direction.
      - calpaterson 13y ago
        I haven't looked at OpenSMTPD. This guide does include a lot of configuration, but it's worth mentioning that not only are the defaults pretty reasonable, but that Ubuntu (and Debian) can configure the most important options using dpkg-reconfigure.
      - spudlyo 13y ago
        If you think Postfix is hard to set up, you've probably never had to deal with Sendmail.
  - VLM 13y ago
    SQLite doesn't have all the conveniences of Oracle products or DB2 so that is not a good analogy. I'd define a minimal working system as the distro's MTA set up to BE a smarthost instead of set up to USE a smarthost which is usually not much, and an imap server so "normal" non-mutt using endusers can access it. Everything else is bolt on fun once that basic setup works. Client side spam filtering gets you out of being the middleman, which is awesome if you'd be stuck as the middleman. There's nothing wrong with the article as a display of nearly the largest most complicated system you can design, which is perfectly cool and fun. Personally I'd add commentary to the article about load balancing and proxying and DNS needs more than "yup gonna have to do DNS" but it is very near a maxed out design. Many HN comments are along the lines of "this maxed out design is large and complicated which implies ..." However That doesn't imply anything about non-maxed out designs. The level of what can be accomplished in (insert programming language) in a maximally complicated situation doesn't necessarily imply much about the difficulty of "hello world" Exim and a simple dovecot imap are about ten lines of config editing total. It won't do much, but sometimes that exactly what is needed.
  - gaadd33 13y ago
    I run a mail server with no spam filtering and its not too bad, maybe 2-3 spam messages a day and Thunderbird catches those. I just have a somewhat strict postfix configuration with graylisting enabled. My email is only ~13 years old so I'm not sure how bad it is for people with older domains/emails.
    - clarry 13y ago
      Spam can be hard to predict. I know people who have had a setup like that (working just fine as for you), and then all of a sudden they've been overwhelmed with hundreds to thousands of spam mails per day.
Sssnake 13y ago
It really isn't. It is a testament to how many people don't want to learn anything, and just want to follow a list of directions. Setting up your own mail server is trivial, it is a 5 minute job.
clarry 13y ago
A mail setup does not have to be this complex. Mine, for example, consists of seven lines of configuration for the daemon, one line to have the init system start the server at boot, plus a few aliases & virtual users. This is using the mail server that comes pre-installed with my OS. It took a few minutes to set up and test. And it's been running for a few years now with no changes.

darklajid 13y ago

This looks like _a lot_ of work was put into the article. Nevertheless, I think you shouldn't do this. This takes a lot of time and is basically copy/pasting things around. For something as crucial as your own mail server I'd go with a solution that is automated: If your server goes away you fire up a different one in a couple of minutes, restore backups and change DNS records, done. Reasons for this might be, in no particular order: - VPS provider goes away - hardware failure - user error (think "Oh, I probably shouldn't have issued that command as root on my mailserver") - maintenance: This guide is (initially) from 2009. How are you making sure that your system is up to date, after you followed the instructions/installed it? Either you'll leave the system to bitrot or you need to actively maintain it and probably will run into issues upgrading the machine. Or .. you migrate to a brand new machine every 6-12 month, which gives you a decent excuse to verify that your backup/restore process is working as well.

nsomaru 13y ago
Considering that this can be installed to a VM, an image of which is backed up, what is not automated about this solution? Maintenance: can't the distro package manager take care of that for me? Sorry for the ignorance in my stance, but I came across this article because I'm looking to move away from Google infrastructure to hosting my own stuff, and this seemed like a great piece :)
- Goladus 13y ago
  VM Images not very portable, and have essentially no built-in documentation. They're also large if you're not working from a snapshot system. Basically, if the exactly prescribed restore procedure fails, a sysadmin is stuck with a virtual black box of bytes and will have to divine out what sort of configuration mismatch there might be between the image and the new host. Different block devices? Different network devices? Wrong CPU architecture? Wrong virtualdisk format? Some other magic of some sort? It can be awhile before you even get the machine up and running, to say nothing about running services. In contrast, a scripted install starts with a running, networked machine and if any point in the install procedure fails, you'll know what point it failed will have a detailed error message explaining why. You can read the script to see what it's trying to do and may be able to either correct the script for the new environment or correct the environment for the script. You can make steady progress towards restoration while if the VM restoration fails, you just keep slamming your head into the brick wall until it gives way. It's true in many situations that restoring a virtual machine snapshot is much faster and easier than a rebuild and data-restore. My experience is that it's great when it works, but a nightmare when it doesn't.

300 13y ago

Respect for all the work done - this is a long but detailed guide!

jvehent 13y ago

I wrote something similar, that uses Postfix, Dovecot, OpenLDAP, Roundcube, DSPAM and OpenDKIM. http://wiki.linuxwall.info/doku.php/en:ressources:dossiers:nectux http://wiki.linuxwall.info/doku.php/en:ressources:dossiers:n... I presented it at my local LUG a while back, here are the slides http://www.slideshare.net/jvehent/di-ymail http://www.slideshare.net/jvehent/di-ymail

ivan_ah 13y ago

Does anyone have experience running a mail server on a home connection with a dynamic IP? I'm imagining a system that will have a homebox (long term storage, privacy) and a cloudbox (provides availability) and the mail flow will be like this: sender ---(1)---> cloudbox ---(2)---> homebox Assuming cloud stays up (1) will happen, and if homebox is reachable, then (2) will also happen. However, if (2) can't be done, then cloudbox will temporarily hold the email until homebox comes online (POP mail style). Has anyone ever setup something like this before? Any pointers will be appreciated.

e12e 13y ago
I've not tried the above -- but there shouldn't be any problem with it (although if you control both servers, you might as well not use smtp from 1 to 2). If you want to do this, you could also just try and keep your dns ttl reasonably low, and updated, and recieve stmp straight at home -- after all: "The sender MUST delay retrying a particular destination after one attempt has failed. In general, the retry interval SHOULD be at least 30 minutes; however, more sophisticated and variable strategies will be beneficial when the SMTP client can determine the reason for non-delivery. Retries continue until the message is transmitted or the sender gives up; the give-up time generally needs to be at least 4-5 days." (My emphasis) http://www.ietf.org/rfc/rfc2821.txt http://www.ietf.org/rfc/rfc2821.txt
VLM 13y ago
This is pretty well trodden ground, between people here and google of decades of history you'll have no problem. Don't forget to implement the arrows flipped around, I sent all my outgoing thru my cloud server after authentication, so everything goes in and out the same way. Eventually I gave up because in my situation forward everything to gmail is not an issue for me; email is just a spam and amazon receipt delivery service, so its not worth much effort. Anyway one interesting design aspect is you're going to fetchmail from home to a cloud pop server and you'll want to secure that, so the natural inclination is to take smallest possible steps while testing all the way, first you try an unusual port number to keep scanners away, maybe you only allow the big netblocks from your provider (not specific /32 addrs but if your provider gives you an addrs from a /19 well then permit the whole /19) then all manner of SSL/TLS protocols instead of plain text passwords and things like that. But I'd advise saving a lot of time and going big for security before even playing with the mail system. So put openVPN on the cloud provider and have your home and your phones/tablets/laptops/friends/family can VPN into your big happy psuedo-internal LAN... don't let "the internet" connect to your cloud pop (imap?) server at all, only allow connections from internal (via VPN) addresses. You're probably going to end up doing it sooner or later and a VPN'd config obsoletes quite a bit of internet accessible stuff for the pop server. You can have a perfectly static internal LAN addrs for homebox and cloudbox over the VPN, even if endpoint addrs change, which makes some simple ip address firewall rules easier than if both endpoints were dynamic. So by the magic of openvpn 10.1.0.0/16 is in the cloudbox and 10.2.0.0/16 is at home, all the time forever, regardless of current provider both at home or cloud, which makes it pretty easy to create static iptables rules and such. I find it handy to have my phone and tablet always on my home network, because I have fun stuff on my home network. So I'm going to be doing this anyway... may as well use the same security infrastructure to secure my fetchmail sessions rather than trying to secure them on the open internet.

blakesterz 13y ago

I've been doing web/mail hosting since 2002 and it amazes me that it is STILL this hard to do a mail server. I gave up and started using iredmail 2 years ago, so far it's been great.

naterator 13y ago

https://github.com/al3x/sovereign https://github.com/al3x/sovereign

csense 13y ago
I've been looking for a good introduction to Ansible -- this may be it!

adders 13y ago

I would definitely recommend using Virtualmin, its a hosting control panel that can control mail/websites/dns in one via a simple control panel. All works like Plesk, but without breaking everything. They've got a script that sets up dovecot, postfix, apache, bind, clamav, spamassassin, Mysql, and loads of other hosting software and then you configure every thing via a web interface. Linux install script is at http://software.virtualmin.com/gpl/scripts/install.sh http://software.virtualmin.com/gpl/scripts/install.sh, instructions at http://virtualmin.com/download http://virtualmin.com/download

plg 13y ago

What do HNers think of the Mac OSX server solution? Assuming you have an apple machine, isn't it rather easy to set up a mail server?

protomyth 13y ago
Yeah, last I looked its dovecot and postfix for OS X. It is a pretty easy setup. Just buy server from the App Store and follow the instructions.

N0RMAN 13y ago

Is it recommended to use a relay for postfix (for private usage) like Mailgun?

spindritf 13y ago

I run my own e-mail server and it's a lot of work. I have exim with ACLs, spamd and just mutt. Nothing "fancy" like other users, pop3/imap/smtp auth, virus scanning, additional mxs... Still took me a weekend to get ACLs right, tune the threshold for greylisting (I greylist e-mails which score between two and seven points in Spam Assasin), set up DKIM and SPF... it's not difficult but there's a long list of things to do. If you're doing it just for yourself, exim, spamd and mutt are enough. But I wouldn't want to be responsible for a larger installation with regular, non-technical users. It's not just the configuration, your service will always be subpar. At least until there's a good webmail (I have high hopes for Mailpile).

strumptrumpet 13y ago
I run my own mail server and it's essentially zero work. I have postfix, dovecot, SpamAssassin, and RoundCube, all authenticated via LDAP. It required some upfront configuration, but it has run itself with minimal configuration changes and 'apt-get' updates for almost 6 years now. I never touch it, and it works. > It's not just the configuration, your service will always be subpar. At least until there's a good webmail (I have high hopes for Mailpile). That assumes that user's want to primarily use webmail. In this renaissance of Mac OS X mail clients and mobile phones, that hasn't been the case here.
- spindritf 13y ago
  LDAP does make things a little easier but how do you synchronize stored e-mails and filtering rules between servers?
  - rhizome 13y ago
    Why do you need to?
  - ibotty 13y ago
    when using dovecot you can use dsync: http://wiki2.dovecot.org/Tools/Dsync http://wiki2.dovecot.org/Tools/Dsync

raimue 13y ago

Good tutorial with detailed set up instructions. On some configuration values I would say the default would suffice as well, but mentioning what can be configured might also be helpful. However, from experience, setting up a database is overkill for a small mail server instance hosting only a few mail boxes. It's way easier to use system user accounts (just disable remote login for them). That avoids the hassles of setting up the virtual mail delivery. Only for a large mail server using a database backend would make sense to gain more performance.

zobzu 13y ago

I run my own email since about 20 years. My current setup is different from the author and I like it better. I had his setup a few years ago I think. - postfix - Dovecot - RoundCube - SpamAssassin w/ sa-learn sa-update and razor - Clamav + Clamav-milter - MariaDB Spam: Not an issue. Features: DKIM, DMARC, SPF checks Maintenance: what maintenance? I update the programs with my distro package manager and that's basically it.. Whoever talked about this being hard.... Webmail: I like RC much better than gmail, nuff said.

mikevm 13y ago
What about the reliability of your home Internet connection? Isn't that a major issue? You can have a power outage, ISP problems, or your server might simply die. What happens to the important emails you might miss?
- nacs 13y ago
  Mail servers are made to attempt redelivery if a mail is undeliverable. Most mail servers will attempt redelivery for multiple days before disposing of it so it shouldn't be an issue.
  - dingaling 13y ago
    Furthermore, to address the 'server on fire' issue. If you build the mail server and its ancillaries up in a VM ( or number of VMs ) then you can sync VM to another host on a schedule, or even live. Set-up a heartbeat ping and the alternative host can bring-up its copy of the mail server VM in a few seconds.
- zobzu 13y ago
  No big deal, its pretty reliable, but also, but i got an MX at a friend's place. Never lost a mail, even if my server goes down for a couple of days. Mail actually retries for a certain amount of time by default, so short downtime generally causes no problem at all. Longer ones do require the 2nd MX, but yeah, I do that. In about 15 years i think the backup MX helped twice so far (which is probably totally worth it) the mail itself is also replicated there in fact

dschiptsov 13y ago

Few hours of work: sudo apt-get/yum install postfix dovecot openldap roundcube.) OpenLDAP schemes and ssl certs are the bulk of it. Hint: never put postfix's spool and dovecot's storage on the same physical device and I/O controller. /var/log must live on the separate device too.

brdrak 13y ago
> never put postfix's spool and dovecot's storage on the same physical device and I/O controller. /var/log must live on the separate device too. This probably depends on the mail volume, no? I host all mail related services for myself and some family on a $9/month VPS (1 VCPU, 512 MB RAM, and pedestrian I/O capabilities) without any issues. My mail volume is typically under 20K messages/day.

borplk 13y ago

The problem with things like this is, since I'm not super confident with any of the technologies involved, it leaves me with an uneasy feeling of having messed something up somewhere. It is extremely easy to set or forget one extra parameter or something and everything falls apart.

ams6110 13y ago

I'd have a look at one of the newer kids on the block, OpenSMTPD. Being from OpenBSD, the man pages are actually good. http://www.opensmtpd.org/manual.html http://www.opensmtpd.org/manual.html

sliverstorm 13y ago
Ugh. I want to use OpenBSD- alas, OpenVZ...

bratsche 13y ago

Out of curiosity: people who are running their own mail servers for multiple users, where do you host?

egoitz 13y ago

I would change that Courier or any Dovecot with a Cyrus IMAP implementation... it really rocks in terms of stability (and works smoothly), is fast and replication :)

houk 13y ago
Really? My experience has been the complete opposite. Almost every time I've even been involved with or ran any medium/large mail infrastructures (200K active mailboxes) I've always found Dovecot to be a better choice.
- brdrak 13y ago
  Good to hear that Dovecot scales. I personally went with Cyrus because at the time I was picking an IMAP server, Cyrus was already a well tested solution known to scale. As I recall, Dovecot back then was still young and developed by one person. I've heard a lot of good things about it since though. I'm sticking with Cyrus since it works fine for me, but it's good to have options.
brdrak 13y ago
Another satisfied Cyrus IMAP user here. :-) I've heard nice things about Dovecot too, though.

exratione 13y ago

I moved off Abrahamsen's recipe 18 months ago or so for no good reason other than I like to make work for myself: it had been working just fine for me. You set it up on a cloud server, take an image, and that's that - you have a working base from which you can restore your mail server if anything bad should happen. Last year I put up the Dovecot / Postfix / Postfix Admin / Horde recipe I used in place of Abrahamsen's, and judging by the feedback it's helped a great many people: https://www.exratione.com/2012/05/a-mailserver-on-ubuntu-1204-postfix-dovecot-mysql/ https://www.exratione.com/2012/05/a-mailserver-on-ubuntu-120... (You can replace Horde with Roundcube or any other webmail package you care to put in place if you don't like it). The market here is for learning, not necessarily for setting up. If you don't know how to set up a mail server, it is well worth walking through one of these as it will teach the lay of the land. Then you can later move to a packaged solution, as you'll be knowledgeable enough to troubleshoot it when it breaks. I was intending to put together a Chef cookbook for my recipe, but looking at what's already out there it seems like it would duplicate some good ones that already exist. e.g.: https://github.com/onddo/postfix-dovecot-cookbook https://github.com/onddo/postfix-dovecot-cookbook I think that running your own mail server is certainly worth the initial effort; once you have the thing set up it ticks along with very little upkeep needed.

shabbyrobe 13y ago

I have tried this guide and others a couple of times. Unfortunately, no matter what I do, Gmail and Hotmail always classify anything I send from it as spam. DKIM? Check. SPF? Check. Reverse DNS? Check. Spamassassin score? 0.0. Result? "google thinks this message is spam". While it was an interesting experience, the end result is invariably a server that can't send mail to most of the people it needs to, which is quite frustrating.

Kototama 13y ago
Here some links that may help you: http://www.brandonchecketts.com/emailtest.php http://www.brandonchecketts.com/emailtest.php http://mxtoolbox.com/ http://mxtoolbox.com/ http://www.codinghorror.com/blog/2010/04/so-youd-like-to-send-some-email-through-code.html http://www.codinghorror.com/blog/2010/04/so-youd-like-to-sen...

jwatte 13y ago

I remember actually understanding sendmail.cf syntax. Those were the days! The problem is not the daemons; those are easy. The problem is spam on the inbound and deliverability on the outbound. And I do /not/ want to fight that fight on my own!

brdrak 13y ago
I've personally not found this difficult. Unless perhaps you're dealing very large volumes of mail and overhead of SpamAssassin is an issue. Mail volume in 10s of thousands/day on a low end VPS has been no problem for me.

voltagex_ 13y ago

How do you handle cut-over to your server when waiting for the DNS changes to propagate?

dirkdk 13y ago

Can I ask the obvious question Why? Email (IMAP/SMTP) is so standard, there are lots of companies out there that do it better and cheaper than you yourself can do. GMail (Google Apps for business) and Fastmail.fm are the 2 names I always recommend. Another factor to consider is how you will keep your SMTP ip-addresses from being blacklisted. Companies like Mailchimp and Sendgrid spend a lot of time and effort on this. With your own very low volume, it takes only a couple of persons to mark your email as spam and suddenly all mails sent from your domain end up in in spam folders, no matter what DKIM or SPF settings you have configured. Only reason I would see is when you want to keep your email away from the NSA and therefore out of the USA (Fastmail is Australian based btw http://thenextweb.com/insider/2013/10/07/are-overseas-based-companies-free-from-nsa-requests-australias-fastmail-thinks-so/ http://thenextweb.com/insider/2013/10/07/are-overseas-based-...). Keeping your server out of the USA will not mean that your email will never travel through the USA, where the NSA can still access it, as SMTP is very cleartext between MTA's.

ivanstojic 13y ago
At the risk of sounding like another one of the "Google is evil" people, I'd like to voice my concern that Google is better at managing mail then I am. I just moved from one country to another and I can't change the billing settings on my Google Apps account because... Google Apps don't support that. The official stance is that I should just download a backup of my accounts, delete them, recreate them with different billing options. However, other than preserving my mail and Google Drive, this won't preserve any of my other content which must either be backed up using third party tools and then manually recovered/recreated. I'm not sure how this can be interpreted as a sign that Google can do this better than I can.
Goladus 13y ago
As much as the NSA concerns me, and as a political issue it's far more important, I am interested in reducing dependency on Google because as far as I am concerned Google has broken its trust with me all by itself. It has nothing to do with the NSA debacle. I don't have time for a rant but the short is: I'm sick of their identity management, their willingness to put blatantly obvious "search bubble" results based on recent (or even non-recent) activity-- stuff that is prominently visible to anyone who might be sharing a screen with me. Ditto for chrome's "most visited pages" thumbnails when you open a new tab, which I disabled but had to install a plugin to do. There's some other minor issues. But the final straw came a few weeks ago when my search history showed up unexpectedly on my girlfriend's phone-- I suppose because she logged into chrome on my laptop to check her Gmail. Until she logged out, auto-complete on her phone was using MY search history from chrome on my desktop. What the FUCK? Obviously I trust her enough to let her log into my laptop so it didn't cause me problems here, but I had no way to know that would happen. So what's it going to be next time? What's google's next clever/reckless silicon valley echo chamber idea that's going to cause me problems? I'd rather not wait to find out. So I've begun the process of reducing google dependence. Email is a big one, Gmail is my single biggest google dependency.

z92 13y ago

save.

locusm 13y ago

Comprehensive, well written article.

robomartin 13y ago

This is really cool. Amazing work. I am looking at running my own mail server out of a Linode instance. It would service a number of our domains and serve to migrate from having web and email live on the same machine at other VPS's. I want to have email serviced from a dedicated instance that does nothing but email for all of our sites. This seems like a better overall strategy but it's been hard to get motivated to get it done due to the amount of work this represents. I have not studied this guide in detail yet but it seems to really take you down the path step by step very nicely. Thanks!