11 ms·
See how DMARC, SPF, and DKIM work interactively
- thyrox 5y agoThis is content marketing done right. You taught me something first in exchange for visiting your service, should I ever need it. I'm fine with such form of advertising.
- deleted 5y ago[deleted]
- peppermint_tea 5y agobest tester I found so far... very detailed, explains interactively what it is doing and gives you the RFC number... was in the process of migrating my mail server, and it reminded me about my spf record :) well done and keep the good work.
- freddieleeman 5y agoThanks, appreciate it!
- deleted 5y ago[deleted]
- CliffStoll 5y agoCool ... A very sweet, understandable demo!
- Karupan 5y agoThis is fantastic! I've always wanted to get a more in-depth understanding of how these protocols are used in the real world, and this was a great format to explore that.
- jeffbee 5y agoWell, a real-world understanding of these protocols would include the knowledge that of the three, DMARC is the least important. Most people do not really need to know about it. If you send valuable transactional emails and you need to prevent or at least reduce the ability of third parties to impersonate you, then you might want DMARC.
- AlexCoventry 5y agoThat's interesting. The current top comment implies that spam filters weigh DMARC so heavily that it's essential to reliable email.
- jeffbee 5y agoThat doesn't make a ton of sense, since the point of DMARC is to stop messages from being delivered. Certainly if you have a DMARC policy of p=reject and then you screw up your outbound, then your DMARC policy becomes relevant, but not in a good way. I don't see the harm in not having one.
- wila 5y ago> I don't see the harm in not having one. Email providers such as outlook and gmail think otherwise. Your emails will not be reliably delivered to them because you are missing a restrictive DMARC policy. Emails send without a DMARC policy are less trusted by default and more likely tagged as spam.
- jeffbee 5y agoThat doesn’t sound right to me. Gmail itself lacks a restrictive DMARC policy. They use p=none.
- peppermint_tea 5y agopeople cannot send spoofed emails with my domain. I'm just an enthusiastic running his mail server so no big gain. But I can see why $BIG_BANK would like to prevent people to be able send fishing email from accounts@$BIG_BANK.com from random computers on the internet...
- sigzero 5y agoThat is pretty cool.
- werdnapk 5y agoFor general information on these protocols, this is a useful starter: https://www.alexblackie.com/articles/email-authenticity-dkim-spf-dmarc/ https://www.alexblackie.com/articles/email-authenticity-dkim...
- iou 5y agoReally cool! I did have trouble with the results aligning correctly on mobile.
- dogmatism 5y agoI host email for my domain at fastmail, but otherwise haven't set up records on it myself. It's amazing any of my mail is delivered at all.
- p2t2p 5y agoFastmail tells you very explicitly what to do and how to do that, I recommend doing that, at one day you may be hit with a very important email going undelivered.
- AndrewDavis 5y agoIf you point to Fastmail as your name server it will even setup the records for you.
- climb_stealth 5y agoHave you tried it with this service? My fastmail managed custom domain doesn't have DMARC set up according to this site. The other two are set.
- EveYoung 5y agoI'm using the recommended Fastmail setup with my own name server, which doesn't support DMARC out of the box. Is this something I should look into to improve deliverability?
- BenjiWiebe 5y agoDMARC is just a TXT record. Which DNS server doesn't support TXT records out of the box?
- EveYoung 5y agoI meant that the recommended guide doesn't include anything regarding the DMARC setup. Technically, it should be possible of course.
- kjs3 5y agoUnfortunately, it's just easier for the big ESPs to ignore/drop email from the self-hosters than deal with it. Which sucks, alot. N.B. - I used to work for a Very Large ESP.
- sbuk 5y agoIP reputation is a different thing. I assume that this is why ESPs tend to drop mail from self-hosters. Low/no reputation or domestic IP often signal malicious activity, so the big ESPs act accordingly. The later reasoning comes from the fact that a vanishingly small amount of email comes from domestic IP addresses, and the stuff that does is often malicious. NB - I work for a large messaging security company.
- kjs3 5y agoI'm not exactly sure what argument you are making. Yes, ESPs often drop most self-hosted email because that's easy and economical (if, perhaps, dickish). No, if you're an international ESP (like mine) there is no real 'domestic IP' you can block against.
- upofadown 5y agoOr in other words: big ESPs have a problem with false positives... which is a known problem.
- kjs3 5y agoOh sure...no argument there. But it's a scale issue. In this space, I have, say, 500M 'questionable' IP addresses who potentially send/receive mail (known bad, residential networks, uncategorized, etc.). So let's be wildly generous and say 0.1% of them want to host an SMTP server. Does the ISP commit resources to this tiny community, or do they commit those resources to keep Google/M$/Yahoo/etc happy, who constitute 9-something% of the business.
- slt2021 5y agois there a tutorial how to create a similar interactive console demo? is it react or angular? Looks very nice!
- freddieleeman 5y agoIt is actually custom build
- jcubic 5y agoIf you don't mind using jQuery Based Terminal[1] library. You can do almost everything (except I don't know how to create floating domain animation). You can think about jQuery Terminal as a framework. You don't need ReactJS or Angular for this, unless you need this as part of bigger application. The library supports typing animation and changing colors. You can easily create the same interactive console. If you're interested I can create a demo for this. [1]: https://terminal.jcubic.pl/ https://terminal.jcubic.pl/
- francispbaker 5y agoInteresting, Comcast is blocking this site. Nice site - I feel like it doesn't good a job at teaching these just validating, which is fine just the name is kinda misleading.
- freddieleeman 5y agoYou can also use https://DMARCtester.com https://DMARCtester.com
- svdr 5y agoThis is very well done, but it incorrectly failed validation of my DKIM signature. I often use https://www.mail-tester.com https://www.mail-tester.com.
- freddieleeman 5y agoPlease share your email details with info@uriports.com and we'll have a look to see what is going wrong.
- raegis 5y agoI don't have any of these on a mail server I've run for 20+ years. Mail still works reliably (both send and receive), but all the naysayers are beginning to make me think about converting to GSuite or similar.
- ckastner 5y agoIt's not just about your own server sending and receiving mail. If you've ever gotten loads of mailer-daemon bounces for emails that you didn't send, then a spammer has been using your address(es) in the "From" header. This eliminates that problem.
- peppermint_tea 5y agono need to... but it is not that hard to implement...
- emilsedgh 5y agoOne of the reasons for your success is the fact that you've been doing it for 20+ years. Big providers like Gmail, Outlook, etc all have a scoring system for sender IP addresses and hostnames. The longer your IP and hostname behaves properly, the more credit it will get. I imagine starting a new mail server with new ip addresses/hostnames may be more difficult.
- anderspitman 5y agoIf you're happy with the way it's working and it's not too much overhead for you to administer it, please don't take down your server. The fewer independent email servers out there, the easier it will be for the big boys to justify no putting in effort to interoperate with the ones that are left.
- technion 5y agoEmail is in a funny space. Spf showed up. And although it meant some political fights when people expected to send email from anywhere, it was free to run in the dollar sense. Same with dkim. Dmarc is increasingly required to get past spam filters. And doing dmarc properly requires a reporting service. Either you run servers or use a hosted service like the one that created this page. That's fine for some business, but putting a cost on dmarc reporting and then making dmarc a heavy signal in spam filters is a huge pain for some scenarios. It's not highly expensive, but on non monetized side projects it can frustrating how how much the internet expects you to have dmarc on every domain now.
- judge2020 5y agoDo you have any personal experience regarding the DMARC requirement claim? AFAIK just having SPF and DKIM without dmarc rua or ruf is still enough to not get hit by Gmail and Office 365 (as in, they’re enough to have your email reputation reliably tied to your domain instead of the email providers having to base a spam decision on the sending IP’s spam reputation).
- technion 5y agoUnfortunately I've had many related experiences. Mail works at first or on a small scale, but eventually a customer with Google workspace mail tells you email goes to junk, and Google support sends you this article in response which describes a range of requirements including dmarc. Same story for office 365. https://support.google.com/mail/answer/81126?hl=en https://support.google.com/mail/answer/81126?hl=en
- ozim 5y agoJust a note that SPF/DKIM and DMARC are not "get out of jail free card". Google support is just brushing people off with the link and they are somewhat right, because a lot don't have anything set. There is also a bit of black magic with spam filters that Google support will never tell you about because it will be just like SEO, someone will use it to pass filters. That said, there are multiple other reasons that your email can be qualified as spam by the spam filter. Like sending out email from one domain and having links to another domain in the body as an example. Not having "unsubscribe" links on mass mail. There are legitimate reasons to throw email to spam because some companies get their infra or "contact us" forms hacked. In the end users can be annoying because they might not understand what "this is spam" button means for the sender - they just think your mail was annoying and they did not expect it and press that and you are qualified as a spam.
- blattinum 5y agoI've modified SPF and set up DKIM and DMARC for my organization and 2 other organizations we merged with in the past year. They're fairly small-sized, but it was definitely a fascinating learning experience if not a bit of a pain. I ended up using Postmark for RUA reporting.
- blattinum 5y agoOh and interestingly I had to set a separate transport rule in M365 in order for M365 to follow our DMARC policy for internal emails. A bit weird, but there you go.
- iJohnDoe 5y agoExtremely cool. Thanks for making this.
- warner25 5y agoQuestion for the experts, as I haven't yet seen a good explanation of this: Why are both SPF and DKIM required to prevent fraudulent emails? My semi-naive intuition is that maybe SPF alone is not sufficient, because of IP address spoofing, but DKIM alone should be sufficient if mail servers adequately protect their private keys. Does SPF just help in the cases where private keys are stolen? Maybe something about DNS cache poisoning, since that's where recipients get the DKIM public keys?
- anderspitman 5y agoI think part of it is that the presence of either is sufficient to validate a sender, but the absence of either doesn't tell you much, because there are multiple valid reasons either could be missing from a valid sender.
- kro 5y agoWhere does it say that? As per rfc7489 4.2: A message satisfies the DMARC checks if at least one of the supported authentication mechanisms: 1. produces a "pass" result Personally I did not test yet what a DMARC result 1 fully failing check produces. I can say however that only 1 successful identifier-alignment criteria (and the other credential being valid but not aligned) is enough for an overall pass. Of course the receiving mailserver is free to enforce a strict DKIM&SPF requirement aside the DMARC check/spec. If your question is why that would be necessary, I'd love to hear the reason aswell
- pracer 5y agoAs mentioned above, I think it is due to the fact that SPF may be too broad, as in softfail SPF (defined here https://datatracker.ietf.org/doc/html/rfc7208#section-2.6.5 https://datatracker.ietf.org/doc/html/rfc7208#section-2.6.5). Based on the RFC, for the rest of possible SPF configurations (more strict), SPF record would be enough. But I am guessing based on the RFC document and my experience.
- peppermint_tea 5y agospf : the receiving server can check (in a public dns record) which server is allowed to send for this domain dkim : the email is signed (using a private key) with a unique signature, the receiving server can validate the DKIM signature by running a DNS query to search for the public key for that domain . dmarc : publish a policy on how emails that does not satisfy the above condition above (spf & dkim) should be handled (quarantine, reject) and optionally sends you a report on who send/tried to send emails on the behalf of your domain Edit : I misread your question... it is odd indeed that only satisfying one condition is enough for a pass...
- waynesonfire 5y agonot very in-depth. i walking in knowing that these things exist to get e-mail to work and require DNS, I walked away with no new insight.
- newaccount74 5y agoHas anybody else been getting spam from "security researchers" and "ethical hackers" who tell you that you are missing a DMARC record and then ask for a bounty?
- guessmyname 5y agoYes. I was the lead security engineer at my previous job, and these messages were always marked as “Informative” and, eventually, “Duplicate”. The major bug bounty websites like Bugcrowd and HackerOne have rules about duplicate and non-actionable submissions, which usually hurt the researcher, for example, by decreasing their reputation points. After the first couple of submissions, I got curious, so I decided to open a separate category for these submissions where I asked every researcher to provide a proof of concept of the attack. Unsurprisingly, none of them were able to. Some of them closed their own submissions, and the others lost some reputation points due to Bugcrowd or HackerOne’s intervention. Somehow, we did not receive more submissions about DMARC, SPF, or DKIM after that, which was interesting, as if all the researchers were working together to spam as many bug bounty programs as possible with the same “vulnerability” to get easy money from unsuspected program managers.
- Brajeshwar 5y agoYes, my domain is one such on the recipient end.
- 2Gkashmiri 5y agoi am using mailinabox.com i sent an email and the result was "Final verdict: No specific action is taken by DMARC regarding the delivery of the message. This usually means the message will be delivered successfully. Keep in mind that other mechanisms such as a spam filter can still reject or quarantine a message." make note that it took me no more than an hour at most in my first attempt to set up the email server including learning about dmarc, spf and where to put that into my email server as well as in the domain registrar. i have had no problems for the past almost a year and i am confident there shouldn't be many problems in future.
- BOOSTERHIDROGEN 5y agoDo you put adkim=r; aspf=r; pct=100, rf=afrf; fo=1 ?
- 2Gkashmiri 5y agoi honestly do not remember. i need to check my registrar. i will do that and get back to you
- freddieleeman 5y agoHere you go: https://www.uriports.com/tools?method=dmarc https://www.uriports.com/tools?method=dmarc
- 2Gkashmiri 5y agofrom https://www.uriports.com/tools?method=dmarc https://www.uriports.com/tools?method=dmarc i go the following result v=DMARC1; p=quarantine
- a1371 5y agoQuestion for HN: how do you handle DMARC reports? We have an email where those are just pulling up in a S3 bucket. I was hoping to find an easy solution that unzips them, aggregates, visualizes, and alerts us (if necessary). In the meantime, I'm hoping to avoid yet another subscription service. Any suggestions?
- khorne 5y agoI have been using https://domainaware.github.io/parsedmarc/ https://domainaware.github.io/parsedmarc/ and throw the DMARC reports into elasticsearch.
- illuminated 5y agoI'm trying to find out a similar solution these days and the best write-up on the topic I have found is here: https://debricked.com/blog/analyze-and-visualize-dmarc-report/ https://debricked.com/blog/analyze-and-visualize-dmarc-repor...
- remus 5y agoIt definitely fails the "avoid another subscription service" criteria but I've found dmarcian works pretty well for me https://dmarcian.com/ https://dmarcian.com/ I was considering setting something up myself but it just seemed like a lot of hassle.
- svdr 5y agoThis is very well done, but it incorrectly failed validation of multiple DKIM signatures.
- xmddmx 5y agoIt also failed DKIM signature validation for me on several accounts, but when I cross-checked vs. http://mail-tester.com http://mail-tester.com it worked fine. I think there's a bug in https://www.learndmarc.com https://www.learndmarc.com
- freddieleeman 5y agoPlease share your email details with info@uriports.com and we'll have a look to see what is going wrong.
- spkm 5y agoSame here. A test email to my google account indicates it passed DKIM.
- kinard 5y agoGreat tool, thanks OP for sharing.
- deleted 5y ago[deleted]
- can16358p 5y agoIs there any progress about the displayed from address being incorrect for phishing attacks than the actual sender? For example I get a (phishing) mail from "Someone <someone@facebook.com>" whereas the message isn't from Facebook at all. And my mail client is perfectly fine with delivering it to my inbox and showing Facebook as the sender. If I'm not tech savvy I'd easily fall for it. If I go down the rabbit hole and view all message headers I see it's fake, but why does the mail client show it as if it's from Facebook in the first place? It's a huge attack vector still present for many non-tech savvy users. (For clarity I'm not talking about the sender name which can be anything that is free text. I'm talking about the "apparent" from field which does not reflect the actual sending address and looks perfectly legit as if from @facebook.com)
- upofadown 5y agoFacebook is an interesting example here. If you give them your PGP indentity they will encrypt and sign the emails they send you. So you will know if the email is actually from them. So technically this is a very solved problem. Facebook unfortunately is very much the exception. It is pretty much impossible to convince entities like your bank to sign their emails.
- OJFord 5y agoI don't understand why clients do that either, the information's there as you say, they just assume the normal/'happy path' case always occurs for some reason. In my own (rather WIP, wouldn't really suggest anyone else use it yet) client I display both if different - which should make it clear even if the content isn't stereotypically crap and obvious spam/phishing (to a more technical user). If you read the RFCs, it was intended for use cases like 'from Mr Blah, but sent by his secretary, reply to his secretary'. Primarily legitimately used today almost like that I suppose - with mailing list / support case management type software being the 'secretary'. So maybe I'm half answering my own point - Gmail et al. show `From` because it's usually friendlier to a non-technical user, it's who it's 'really' from, not confusing them with 'who is this Intercom.io' or whatever. But.. surely it's worth showing both, to help in the phishing cases as you say.
- emilsedgh 5y ago
- stiray 5y agoGreat idea, there are lots of others that are showing you if SPF/DMARC/DKIM work correctly but this one is also great for people trying to set up mail server and are confused what is actually going on. Nice work. It would be very nice if more header information would be exposed so you exactly see what is taken into account.
- TheRealDunkirk 5y agoAll these checks for validity, and still, my Gmail spam folder overflows. What was the point again?
- scandox 5y agoIt was previously possible for an attacker, especially a selective one to impersonate the domain of a major provider like Paypal or Dropbox in the display address. I think the fact this is no longer possible is a good thing. Phishing has got a lot more aggressive, and obviously if these measures were not being deployed the situation would simply be a lot worse.
- brightball 5y agoDMARC is one of many, many things that Gmail’s algorithms are taking into account.
- ChrisMarshallNY 5y agoThat's excellent! I tried it with all three of my domains. It's interesting that two of them didn't have full DKIM/DMARC support, even though they are paid mail services. I'll have to check, and see if there is anything that I can do. I seldom have issues with bounces or spam-suckers, but I have had a couple, with those domains. My mac.com domain was green, all the way.
- jp8i 5y ago