The only thing TLS in the mail server can't give you is a way to outsource the running of your domain's MX server without trusting that server with your SSL key

The only thing TLS in the mail server can't give you is a way to outsource the running of your domain's MX server without trusting that server with your SSL key. If the sending server PGP encrypts instead you can decrypt it later. That would make running your own mail a little easier. Use any normal SMTP/POP/IMAP email service to handle the network side and then encrypt/decrypt locally. You only have to trust your own mail user agent running on your own hardware. As it stands right now to run my own email (something I've been contemplating) I have to either put my SSL key in a machine I don't control (e.g. Linode VPS) or try to run a mail server on a residential internet connection. Neither seems a good option.