12 ms·
SPF, DMARC, and DKIM: How to Keep Your Email Out of the Spam Folder
- chewmieser 10y agoI've had great luck with my personal email servers thanks to this tool: https://www.mail-tester.com/ https://www.mail-tester.com/ Gives you a score and suggestions on improving it to reduce the chance of hitting the spam filter.
- ryandonsullivan 10y agoCool tool. I'll have to play around with that and see how it goes. May end up adding it to the article :)
- icebraining 10y agoI like this one from NIST: https://www.had-pilot.com/py/had.html https://www.had-pilot.com/py/had.html You just send an email to the address at the bottom and it replies with a bunch of information regarding its SPF, DKIM and/or DMARC.
- Erwin 10y agoFor something more complete: https://glockapps.com/ https://glockapps.com/ This gives you (in the paid version) a "seed list" of emails at various ISPs, so you can see how your email does with gmail vs yahoo. This is one of the several services, but the one we're most happy with.
- slasaus 10y agoI think it's pretty weak they're advertising the use of "~all" in their spf records. Either use "-all" or just don't use SPF I would say. If you can't make a decisive statement about your own domain then it won't be actionable for receivers that evaluate your records.
- ryandonsullivan 10y agoThat's a totally fair point. I'm not entirely sure why most third parties are still using ~ in their documentation but it still seems to be the norm. I do like the definitive nature of -all.
- deleted 10y ago[deleted]
- aroch 10y agoIIRC ~all is the recommendation because hotmail/live told people to use ~all to prevent hardfails when hotmail's lookups timed out or if a particular mailserver IP was inaccessible during spam checks. ~all will result in your email being bounced around until accepted even if the IP doesn't match DNS records (more or less). -all will result in hardfail if rejected by any TO mailserver.
- Scramblejams 10y ago-all isn't necessarily a slam dunk because it interacts badly with lots of mailing lists. There is standards work underway to improve this, but it's still an issue, for example https://www.ietf.org/mail-archive/web/ietf/current/msg87153.html https://www.ietf.org/mail-archive/web/ietf/current/msg87153....
- medmunds 10y agoIf you use DMARC with a reject or quarantine policy, SPF hardfail ("-all") can prevent recipients from successfully forwarding mail you've sent them. Some best practices for DKIM, SPF, and DMARC (as of mid-2015) in [1], including this: > ...when an organization publishes p=reject [in DMARC], they should simultaneously change their SPF hard fail to SPF soft fail. ... A message that passes SPF and is forwarded will fail SPF. If a message hard fails SPF it will probably be marked as spam but if it soft fails, it will most likely still be accepted by the recipient. This forwarding failure possibility is why most organizations publish a soft fail record. [1]: https://blogs.msdn.microsoft.com/tzink/2015/07/12/what-is-the-best-combination-for-your-spf-record-dkim-record-and-dmarc-record/ https://blogs.msdn.microsoft.com/tzink/2015/07/12/what-is-th...
- slasaus 10y agoInteresting note about DMARC, but still, if you're concerned about breaking forwarding for your domain, then why bother using spf at all? I still don't see the benefit of setting up ~all rules.
- medmunds 10y agoI believe that DMARC requires SPF. Since I want DMARC, I need to provide a compatible SPF, which means ~all. (And I do want to implement DMARC. Not so much to improve deliverability of my own email, but rather to prevent delivery of malicious email pretending to be from my domain.)
- slasaus 10y agoOk, just wondering, would "v=spf1 ?all" be the same in that case? I.e. a neutral spf policy?
- tnorthcutt 10y agoGoogle specifically says[0]: Create a TXT record containing this text: v=spf1 include:_spf.google.com ~all Publishing an SPF record that uses -all instead of ~all may result in delivery problems. See Google IP address ranges for details about the addresses for the Google Apps mail servers. [0]: https://support.google.com/a/answer/178723?hl=en https://support.google.com/a/answer/178723?hl=en
- ryandonsullivan 10y agoThanks for posting this. I know it's on a WordPress site but definitely applies to way more people than just that audience. I hope to build the article out even more and get super specific with various vendors like Mandrill, Amazon SES, Sendgrid, etc.
- djsumdog 10y agoI wrote this a while ago. I have SPF, DMARC and DKIM all implemented on my mail domain and I still get put in the spam folder: http://penguindreams.org/blog/how-google-and-microsoft-made-email-unreliable/ http://penguindreams.org/blog/how-google-and-microsoft-made-... I think part of it might be that I use Linode, and there are other spammers in their data centre, so I could just be on a subnet bad list. But I think a lot of it has to do with Google/Microsoft's spam filters just being crazy over aggressive.
- ryandonsullivan 10y agoYeah, no doubt that a sketchy IP will basically negate any kind of authentication you have in place. Off to read your article...
- ajsalminen 10y agoHave you also set up reverse DNS and done it all for IPv6 too? The second part was what I was missing a while back. I do agree that Google and Microsoft are extremely strict with what they accept and it's not always easy to tell why you get thrown to the spam pile.
- codezero 10y agoYep, I think there was a post about setting up IPv6 correctly on HN a while back. Alternatively, you can make your host not respond to IPv6 at all. If I remember correctly, the problem isn't that IPv6 needs to be set up, but that if your host listens on IPv6, Google will default to that and then your SPF/DMARC/DKIM setup needs to work with IPv6. If you don't have an external IPv6 address, then you don't need to configure SPF etc... for IPv6 and you should be fine.
- rbinv 10y agoIt depends, really. When you're just starting out with sending email from a fresh domain and IP address, you will most likely be flagged no matter what. Do not underestimate signals that come from user interaction (e.g. open and click rates). Deliverability is a reputation game.
- 10y ago
- edutechnion 10y agoA few tips from setting up SPF/DMARC/DKIM for a SAAS service: * SPF: limit your record and all includes to 10 DNS lookups (e.g., "A MX include:_spf.google.com" is 3 DNS lookups plus all of the lookups inside the include. * DMARC: to see a strict reject policy, check out Yahoo: $ dig +short -t txt _dmarc.yahoo.com "v=DMARC1\; p=reject\; pct=100\; rua=mailto:dmarc_y_rua@yahoo.com\;" * Mail forwarding: if your app sends mail as the logged-in user, make sure the user's actual email address is not in the FROM address as Yahoo does not authorize you to send FROM: xxxx@yahoo.com * DMARC emails: use dmarcian.com to parse and process the auto-generated emails * SPF: use the ~all for your first day of testing and then lock it down to -all after testing is complete * DKIM: OpenDkim appears to be the most widely supported Linux software package. * DKIM keys: setup a TXT entry you control and ask client to CNAME it. Then setup key rotation.
- walrus01 10y agoOn the receiving side: Ensure that your world-facing port 25 smtpd is properly pipelining all incoming emails through openDKIM so that their headers are tagged with openDKIM validation or lack thereof. This can work in conjunction with spamassassin.
- zimbatm 10y agohttps://dmarcian.com/ https://dmarcian.com/ is a really nice service. Their support was also very helpful while trying to figure out the same things you did. Since there is a 10 query limit on SPF, if you delegate your SPF to third-parties like Google then your SPF might blow up unexpectedly if they increase the number of records on their side. Dmarcian monitors that for example.
- talideon 10y agoFor DKIM, rspamd/rmilter are a great alternative to OpenDKIM if you want to build the DKIM check into your regular spam checks. One minor downside of rmilter is that it will only sign the headers of mail sent by by authenticated users. This isn't a huge deal, but can be a bit of an irritation.
- cebka 10y ago
- dmuth 10y agoJust to throw this out, I'm the guy responsible for one of the "top 10" DMARC reporting engines on the Internet in terms of volume. If anyone would like to chat about DMARC in the real world or about the DMARC reports (that they are very likely receiving from my employer's domain name), feel free to reach out to me. My email is my username at my username dot org.
- Erwin 10y agoGoogle's Postmaster tools always tell me 100% DKIM, 100% SPF but a crazy varying amount of DMARC success rate. Postmark's weekly report gives me 99.9% DMARC compliance. Do you have an explanation or theory? Our envelope-from often differs from header-From but none of those domains have a strict DMARC alignment policy.
- medmunds 10y agoI believe you can explain some common cases in Postmark's weekly digest like this (but would really appreciate confirmation from an actual specialist in the field): 1. "Trusted sources" (DMARC fully/partially aligned), DKIM pass, but SPF fail: a recipient has forwarded your fully-aligned email. 2. "Untrusted sources" (DMARC not aligned), DKIM fail, SPF fail: genuine spam, or email forwarding that also rewrites headers in way that breaks DKIM (like the recent Hotmail/Outlook.com forwarding problem). 3. "Untrusted sources", DKIM pass, SPF pass: properly signed and SPF'd, but your envelope-from domain doesn't match the header-From domain. If your DMARC policy is reject or quarantine, these messages won't get delivered. One way to get case 3 is with a vendor sending on your behalf, where you've included their SPF in your own record (so SPF pass), but they sign DKIM and set envelope-from using their domain. The DKIM is valid for your vendor, so passes, but doesn't match the From, so DMARC is not aligned and fails. For example, UserVoice has this problem if you're using a custom From address in your domain. And Gmail shows this type of message as "From <you> via <vendor>".
- dmuth 10y agoI can't say without more details, but if you can reach out to me privately, I'm more than happy to dig through logs on our mail servers (well, dig through our Splunk platform...) and tell you exactly what we're seeing from your domain.
- talideon 10y agoIt's worth keeping in mind that the real value of SPF records isn't preventing you from receiving spam (aside from backscatter), but to prevent Joe-jobs.
- deleted 10y ago[deleted]
- cm3 10y agoSay I want a personal domain and have the MX not be Google or Fastmail, how complete is the implementation of these standards on major email providers' SMTP setups? Would I need to do thorough research or is it reasonable to expect it to just work in a, say, European SMTP hoster's configuration?
- walrus01 10y agoAny major ISP with clue is at least checking SPF, DMARC and DKIM scores on its incoming edge smtpd that talks to the world. How they use the scoring and results for spam filtering varies widely.
- deleted 10y ago[deleted]
- cm2187 10y agoI am surprised that almost no one seems to be using DKIM. I tried setting a higher spam weight on unsigned emails and half of my emails ended up in the spam folder. SPF is used more frequently.
- znpy 10y agoI host my own email at home on a domestic connection with a domestic fixed ipv4 and I only set up SPF. Most e-mail providers accept my email. More accurately: in the last two years only gmx.de rejecte one email.
- logicallee 10y agoI don't like the implication that the work outlined in this article is reasonable. Imagine if there were a detailed guide on how to keep the post office from throwing out the letters you send? Because mail that you personally send out by definition isn't spam - so you are doing work to get around broken spam filters. why can't you just pay $10 or something as a deposit and, since you're not actually a spammer and nobody will ever actually mark what you send as spam, never lose that deposit. This guide should be like four lines long and take 5 mi utes to follow. I mean after glancing at that write-up, I'd never dream of running my own mail server. I use gmail. Why would I jump through hoops and still risk having letters I took the time to write, still marked as spam? I lose on two counts! (invest time, for a worse outcome.) This part of the industry is broken. I think a deposit paid by non-spammers which they lose if people start marking their letters spam, might fix it.
- specialp 10y agoI have found that the site https://mxtoolbox.com/ https://mxtoolbox.com/ is very good for scanning your records for SPF/DMARC/DKIM and pointing out problems. Unfortunately even with implementing all of this some email providers are heavy handed with rejecting mail from smaller mail servers. For instance Yahoo will usually block all of your mail without appealing to them, and Verizon will not allow any email originating from a VPS source like Digital Ocean or AWS. Ironically I found that out when going to report someone on Verizon's network trying to brute force my SMTP server. Mail to abuse was rejected.
- unethical_ban 10y agoI resent that IP block based filtering is done anymore. Legit people are using VPS to send email, and email is a fundamental decentralized protocol of the internet. It's really crappy that in this age of DKIM/SPF/DMARC, we can't do away with IP filtering.
- brightball 10y agoSendgrid has a nice implementation for DKIM too. They setup 2 DKIM CNAME records that point to a DNS entry with a id number in it for you. Periodically, they rotate DKIM keys and by having the 2 CNAME's they can easily transition without concerns about something getting lost in transit.
- dtemp 10y ago_domainkey.yoursite.com TXT "t=y; o=~;" Does anyone know how necessary this entry is, as opposed to just having records beginning with selectors? It seems like the t=y means that testing is on and to not actually block messages that fail DKIM, and o=~ means that some messages aren't signed. I'm not sure why the article is suggesting people use those settings, since they are entirely variable between different users and their config.