4 ms·

I posted this two days ago. I believe a moderator must have rescued this post today to bring it to the front page. By the way, I posted this link because I've

by susam 7y ago

I posted this two days ago. I believe a moderator must have rescued this post today to bring it to the front page. By the way, I posted this link because I've been running my own mail transfer agent (MTA) for my personal vanity domain to send/receive emails for about 10 years now and I often forget why I need to edit /etc/mailname while setting up the MTA, so I posted this link to HN as a bookmark for myself. (Who else uses HN submissions as bookmarks?) I use Exim 4 as my MTA on Debian GNU/Linux. During its setup, among many other things, I need to configure /etc/mailname with the domain name I want to see in the "From:" field of outgoing security notification emails, e.g., email notifications for failed sudo attempts. If you want to take a look at my MTA setup, see this Makefile: https://github.com/susam/susam.in/blob/master/mta/Makefile https://github.com/susam/susam.in/blob/master/mta/Makefile Related Show HN post: https://news.ycombinator.com/item?id=22299169 https://news.ycombinator.com/item?id=22299169

hamnspam 7y ago
Is it possible to run own SMTP server these days and send emails successfully? I thought GMail will classify it as spam. How do you tell the big email providers to not classify email originating from personal domain as spam?
- rstuart4133 7y ago
  It's not difficult - most distros have email servers you just install. Most of those use that same email server internally for their mailing lists. It's not much harder (or easier) than setting up any other server. GMail does not classify an "own email server" as source of spam. Quite apart from anything else how would it know it's an "own email server" as opposed to some companies email server or even a distro email server. They all user the same code. To get your email server classified as a spammer you have to send spam (mostly). That won't happen if it is just forwarding emails you create, but if it is the work horse behind an email list server or something most of what the list will see is spam and not relaying that to GMail or something becomes very hard indeed. The reason for the (mostly) above is occasionally somebody will have finger trouble and mark some good email coming for your server as spam. As it happens GMail seems fairly immune to this problem. I guess they must need quite a few people to do make it before it kicks in. Microsoft's servers seem to make the mistake occasionally, and some middle wear boxes can be bloody horrible about it. Fortunately they all seem to provide web pages to fix their mistake you just have to find it.
  - JdeBP 7y ago
    In a universe where the SORBS DUHL and its ilk never existed, this may be true. In this universe, however, they did and do exist, and running things on one's own machines renders one a third-class citizen. * https://news.ycombinator.com/item?id=14719504 https://news.ycombinator.com/item?id=14719504
    - rstuart4133 7y ago
      I'm not sure what evidence you have for this, but I'll provide my own: I run over 30 mail servers on normal ISP links. I can categorically state that's not a problem. FWIW, I agree was a problem years ago. I suspect it isn't now because ISP's can not longer say what an IP Address is used for, or maybe it's because the DU in that RBL's name stands for Dial Up and this is 2020, not 1999. There is one caveat: you do have to get the DNS PTR records set up.
- colechristensen 7y ago
  A good way to avoid this is to setup dkim and spf for your email domain. https://www.sparkpost.com/blog/understanding-spf-and-dkim/ https://www.sparkpost.com/blog/understanding-spf-and-dkim/
- susam 7y ago
  I follow the usual best practices and set up DNS TXT records for Sender Policy Framework (SPF), DomainKeys Identified Mail (DKIM), and Domain-based Message Authentication, Reporting & Conformance (DMARC). Only DKIM requires configuring the MTA for it because the MTA needs to know where the DKIM private key is to sign emails and what the DKIM selector, domain, etc. should be in the DKIM-Signature header. From my Makefile[1]: # -------------------------------------------------------------- # Enabling DKIM ... # -------------------------------------------------------------- echo DKIM_CANON = relaxed >> /etc/exim4/exim4.conf.localmacros echo DKIM_SELECTOR = s1 >> /etc/exim4/exim4.conf.localmacros echo DKIM_DOMAIN = $(FQDN) >> /etc/exim4/exim4.conf.localmacros echo DKIM_PRIVATE_KEY = /etc/exim4/exim.key >> /etc/exim4/exim4.conf.localmacros openssl pkey -in /etc/exim4/exim.key -pubout -outform DER | base64 -w 0 > /tmp/publickey After that, it is just a matter of adding the necessary TXT records for SPF, DKIM, and DMARC to the DNS zone. The relevant section in the Makefile[2]: # -------------------------------------------------------------- # Generating DNS records ... # -------------------------------------------------------------- @echo "Add the following TXT records to DNS:" @echo @echo "@ TXT v=spf1 mx -all" @echo "s1._domainkey TXT p=$$(cat /tmp/publickey)" @echo "_dmarc TXT v=DMARC1; p=reject; adkim=s; aspf=s" Having said that, I have also found in the past that major email providers would accept emails sent by my MTA just fine even without these measures. For a long time, I did not have SPF, DKIM, or DMARC enabled and I did not observe any issues. However, it is a good practice to use these measures to make it harder for a bad actor to spoof and send spam that appears to originate from your domain. [1]: https://github.com/susam/susam.in/blob/7cff99b/mta/Makefile#L48-L57 https://github.com/susam/susam.in/blob/7cff99b/mta/Makefile#... [2]: https://github.com/susam/susam.in/blob/7cff99b/mta/Makefile#L68-L75 https://github.com/susam/susam.in/blob/7cff99b/mta/Makefile#...
- gsich 7y ago
  It is.
- stevekemp 7y ago
  I setup a simple new virtual-server yesterday, initially outgoing mail was rejected at SMTP-time by Gmail with an error referring to reverse-DNS for the IPv6 address. Fixing a couple of things like that (add/update a SPF record, setup reverse DNS for IPv4 & IPv6) and mail was successfully delivered to gmail. Biggest pain there is that some virtual-machine providers don't let you change RDNS for IPv6 addresses (e.g. scaleway). So you get the choice of disabling it, or changing provider.
- jacobush 7y ago
  I never had any problem with gmail, but outlook et al I can no longer send mail to, since I switched VPS providers. My IP range is on a blacklist and it won't come off.
  - pmlnr 7y ago
    Outlook has a manual way of removing a single ip. There should be a URL in your mailer daemon response to do so. And yes, f u outlook.
    - jacobush 7y ago
      Yep, but as far as I understand, the VPS provider should use the URL. And they (supposedly) did, twice and nothing came of it.
      - tomatocracy 7y ago
        If you have the reverse DNS set to something within a domain you own, they'll usually let you do it yourself (the process involves them sending an email to postmaster or abuse at the parent domain of the server or at the AS which controls the IP address, but you get to choose which one). Still doesn't seem to stop them randomly flagging emails as spam, sometimes mid email chain. The MS support team are also responsive in my experience.
- tzs 7y ago
  I've never had trouble sending to gmail. I've never had occasion to send to yahoo mail, but recently sent a test message to an account I had there. Yahoo said it was spam. I marked it as not spam. After that, sending to yahoo was fine. I also had never sent to outlook mail. I made an account and tried sending mail from my domain. It went to the spam folder. I told outlook it was not spam. Tried another message...still spam. Told it that it was not. I think I'm up to 10 or so such messages now, and outlook is still saying they are spam.
- inopinatus 7y ago
  Totally possible. Run SPF, DKIM, and DMARC, have correct forward and reverse DNS, have reliable DNS servers for every lookup, forward with ARC, deliver with a valid certificate, prefer IPv6, and have all associated IP addresses and sender (SMTP envelope) domains with a very good reputation. This last part takes months if not years to build, it’s not hard, but it’s not fast. I use separate sender domains for a newsletter than for transactional mail. You can monitor reputation with Gmail and Outlook postmaster tools, and there are various sites that will mass query RBLs; worth checking occasionally. I have a monthly reminder in the calendar. Sign up for the JMRP and SNDS. Then just behave like a responsible sender. Don’t send unsolicited mail. Make it easy to unsubscribe, with RFC8085 headers and easily found unsubscribe links. Honour unsubscribes instantly. Honour spam reports instantly with a suppression list. Make the first statement of your communication one that describes the purpose; the second, the unsubscribe link. Make it easier, both functionally and emotionally, for recipients to unsubscribe rather than to report you as junk. Run your emails through SpamAssassin before sending, and respect and understand the scoring. Read the Gmail sender guidelines (https://support.google.com/mail/answer/81126 https://support.google.com/mail/answer/81126), they are good advice. Include a plain text part. Do not trust anyone claiming you can pay to improve reputation. That is snake oil.
  - anticodon 7y ago
    No. GMail is arbitrarily blocking ip addresses or subnets and there's nothing you can do about it. One year ago I've purchased a VPS with a clean IP address (checked against all DNSBLs), set up mail server with SMTP and IMAP server, set up SPF, DKIM and DMARC and was able to successfully send emails to Gmail for 3 months. After 3 months GMail stopped accepting mail from my server saying that my ip address was used to send spam. I'm pretty much sure it wasn't me. My VPS OS and mail server are up to date, there was no suspicious activity in the logs, so it's very unlikely that I was hacked. There're only three possible explanations for it: 1. My VPS was hacked indeed 2. Some bad guy from the same IP network sent spam (although GMail error message specifically states that spam was sent by this exact IP address, not an address from the same network). 3. GMail just randomly blocks access to any little guy trying to setup independent mail server just because they can. I'll never know the truth but it was very disappointing experience. Also, lost couple of semi-important emails this way.

by susam 7y ago

hamnspam 7y ago

Is it possible to run own SMTP server these days and send emails successfully? I thought GMail will classify it as spam. How do you tell the big email providers to not classify email originating from personal domain as spam?

rstuart4133 7y ago
It's not difficult - most distros have email servers you just install. Most of those use that same email server internally for their mailing lists. It's not much harder (or easier) than setting up any other server. GMail does not classify an "own email server" as source of spam. Quite apart from anything else how would it know it's an "own email server" as opposed to some companies email server or even a distro email server. They all user the same code. To get your email server classified as a spammer you have to send spam (mostly). That won't happen if it is just forwarding emails you create, but if it is the work horse behind an email list server or something most of what the list will see is spam and not relaying that to GMail or something becomes very hard indeed. The reason for the (mostly) above is occasionally somebody will have finger trouble and mark some good email coming for your server as spam. As it happens GMail seems fairly immune to this problem. I guess they must need quite a few people to do make it before it kicks in. Microsoft's servers seem to make the mistake occasionally, and some middle wear boxes can be bloody horrible about it. Fortunately they all seem to provide web pages to fix their mistake you just have to find it.
- JdeBP 7y ago
  In a universe where the SORBS DUHL and its ilk never existed, this may be true. In this universe, however, they did and do exist, and running things on one's own machines renders one a third-class citizen. * https://news.ycombinator.com/item?id=14719504 https://news.ycombinator.com/item?id=14719504
  - rstuart4133 7y ago
    I'm not sure what evidence you have for this, but I'll provide my own: I run over 30 mail servers on normal ISP links. I can categorically state that's not a problem. FWIW, I agree was a problem years ago. I suspect it isn't now because ISP's can not longer say what an IP Address is used for, or maybe it's because the DU in that RBL's name stands for Dial Up and this is 2020, not 1999. There is one caveat: you do have to get the DNS PTR records set up.
colechristensen 7y ago
A good way to avoid this is to setup dkim and spf for your email domain. https://www.sparkpost.com/blog/understanding-spf-and-dkim/ https://www.sparkpost.com/blog/understanding-spf-and-dkim/
susam 7y ago
I follow the usual best practices and set up DNS TXT records for Sender Policy Framework (SPF), DomainKeys Identified Mail (DKIM), and Domain-based Message Authentication, Reporting & Conformance (DMARC). Only DKIM requires configuring the MTA for it because the MTA needs to know where the DKIM private key is to sign emails and what the DKIM selector, domain, etc. should be in the DKIM-Signature header. From my Makefile[1]: # -------------------------------------------------------------- # Enabling DKIM ... # -------------------------------------------------------------- echo DKIM_CANON = relaxed >> /etc/exim4/exim4.conf.localmacros echo DKIM_SELECTOR = s1 >> /etc/exim4/exim4.conf.localmacros echo DKIM_DOMAIN = $(FQDN) >> /etc/exim4/exim4.conf.localmacros echo DKIM_PRIVATE_KEY = /etc/exim4/exim.key >> /etc/exim4/exim4.conf.localmacros openssl pkey -in /etc/exim4/exim.key -pubout -outform DER | base64 -w 0 > /tmp/publickey After that, it is just a matter of adding the necessary TXT records for SPF, DKIM, and DMARC to the DNS zone. The relevant section in the Makefile[2]: # -------------------------------------------------------------- # Generating DNS records ... # -------------------------------------------------------------- @echo "Add the following TXT records to DNS:" @echo @echo "@ TXT v=spf1 mx -all" @echo "s1._domainkey TXT p=$$(cat /tmp/publickey)" @echo "_dmarc TXT v=DMARC1; p=reject; adkim=s; aspf=s" Having said that, I have also found in the past that major email providers would accept emails sent by my MTA just fine even without these measures. For a long time, I did not have SPF, DKIM, or DMARC enabled and I did not observe any issues. However, it is a good practice to use these measures to make it harder for a bad actor to spoof and send spam that appears to originate from your domain. [1]: https://github.com/susam/susam.in/blob/7cff99b/mta/Makefile#L48-L57 https://github.com/susam/susam.in/blob/7cff99b/mta/Makefile#... [2]: https://github.com/susam/susam.in/blob/7cff99b/mta/Makefile#L68-L75 https://github.com/susam/susam.in/blob/7cff99b/mta/Makefile#...
gsich 7y ago
It is.
stevekemp 7y ago
I setup a simple new virtual-server yesterday, initially outgoing mail was rejected at SMTP-time by Gmail with an error referring to reverse-DNS for the IPv6 address. Fixing a couple of things like that (add/update a SPF record, setup reverse DNS for IPv4 & IPv6) and mail was successfully delivered to gmail. Biggest pain there is that some virtual-machine providers don't let you change RDNS for IPv6 addresses (e.g. scaleway). So you get the choice of disabling it, or changing provider.
jacobush 7y ago
I never had any problem with gmail, but outlook et al I can no longer send mail to, since I switched VPS providers. My IP range is on a blacklist and it won't come off.
- pmlnr 7y ago
  Outlook has a manual way of removing a single ip. There should be a URL in your mailer daemon response to do so. And yes, f u outlook.
  - jacobush 7y ago
    Yep, but as far as I understand, the VPS provider should use the URL. And they (supposedly) did, twice and nothing came of it.
    - tomatocracy 7y ago
      If you have the reverse DNS set to something within a domain you own, they'll usually let you do it yourself (the process involves them sending an email to postmaster or abuse at the parent domain of the server or at the AS which controls the IP address, but you get to choose which one). Still doesn't seem to stop them randomly flagging emails as spam, sometimes mid email chain. The MS support team are also responsive in my experience.
tzs 7y ago
I've never had trouble sending to gmail. I've never had occasion to send to yahoo mail, but recently sent a test message to an account I had there. Yahoo said it was spam. I marked it as not spam. After that, sending to yahoo was fine. I also had never sent to outlook mail. I made an account and tried sending mail from my domain. It went to the spam folder. I told outlook it was not spam. Tried another message...still spam. Told it that it was not. I think I'm up to 10 or so such messages now, and outlook is still saying they are spam.
inopinatus 7y ago
Totally possible. Run SPF, DKIM, and DMARC, have correct forward and reverse DNS, have reliable DNS servers for every lookup, forward with ARC, deliver with a valid certificate, prefer IPv6, and have all associated IP addresses and sender (SMTP envelope) domains with a very good reputation. This last part takes months if not years to build, it’s not hard, but it’s not fast. I use separate sender domains for a newsletter than for transactional mail. You can monitor reputation with Gmail and Outlook postmaster tools, and there are various sites that will mass query RBLs; worth checking occasionally. I have a monthly reminder in the calendar. Sign up for the JMRP and SNDS. Then just behave like a responsible sender. Don’t send unsolicited mail. Make it easy to unsubscribe, with RFC8085 headers and easily found unsubscribe links. Honour unsubscribes instantly. Honour spam reports instantly with a suppression list. Make the first statement of your communication one that describes the purpose; the second, the unsubscribe link. Make it easier, both functionally and emotionally, for recipients to unsubscribe rather than to report you as junk. Run your emails through SpamAssassin before sending, and respect and understand the scoring. Read the Gmail sender guidelines (https://support.google.com/mail/answer/81126 https://support.google.com/mail/answer/81126), they are good advice. Include a plain text part. Do not trust anyone claiming you can pay to improve reputation. That is snake oil.
- anticodon 7y ago
  No. GMail is arbitrarily blocking ip addresses or subnets and there's nothing you can do about it. One year ago I've purchased a VPS with a clean IP address (checked against all DNSBLs), set up mail server with SMTP and IMAP server, set up SPF, DKIM and DMARC and was able to successfully send emails to Gmail for 3 months. After 3 months GMail stopped accepting mail from my server saying that my ip address was used to send spam. I'm pretty much sure it wasn't me. My VPS OS and mail server are up to date, there was no suspicious activity in the logs, so it's very unlikely that I was hacked. There're only three possible explanations for it: 1. My VPS was hacked indeed 2. Some bad guy from the same IP network sent spam (although GMail error message specifically states that spam was sent by this exact IP address, not an address from the same network). 3. GMail just randomly blocks access to any little guy trying to setup independent mail server just because they can. I'll never know the truth but it was very disappointing experience. Also, lost couple of semi-important emails this way.