7 ms·

Email Authentication: A Developer's Guide

209 points by zenorocha 3y ago

PreInternet01 3y ago
Note that this is about the use case 'authenticating whether the given mail server is authorized to send mail for the given domain', not the use cases 'authenticating whether the given user of this mailbox actually sent this email' or 'authenticating whether the given message is spam or not', which is probably the reason you clicked on this article :). The former use case is pretty much solved (in that you can safely ignore email from servers/domains that don't follow best practices), the latter (combining the 2, since they're pretty similar, really) is not, even given recent advances in AI (OpenAI cannot tell you if a message is spam, sorry, unless your prompt engineering skills are much better than mine).
- hashstring 3y ago
  > OpenAI cannot tell you if a message is spam, sorry, unless your prompt engineering skills are much better than mine. Sometimes an e-mail message itself does not even contain enough information to accurately classify it as spam or phish. To a degree, spam is subjective. And classifying a phish may not be trivial at all (e.g. message may include legit marketing links, open redirects and server side logic to serve certain pages only to targets, etc.).
  - bruce343434 3y ago
    Yup, when I order something I get really annoyed that I get an email for every fart that the delivery driver lets out. I'll hear the doorbell, I don't need 500 anticipatory emails. It's not a scam, not a phish, and it's 100% factual and "informative". But still junk.
    - akho 3y ago
      It isn't spam though (being a part of a real commercial relationship, and having a working unsubscribe link), and marking it as such poisons your spam filter.
  - heipei 3y ago
    Thank you for that comment, I don't think it's something many people really understood. The same is true for phishing websites. So much depends on the context and incomplete background information. Is a website that asks you to put in your username and password bad? Well, it depends on what the website does with that information in the background. I've seen very suspicious websites asking for user information which were, in the end, just sites set up by marketing departments of the larger company who were unaware of the dangerous precedent they were setting.
- thijsvandien 3y ago
  To further emphasize the point that a mail server being authorized often isn't worth much, or can be counterproductive even: shocking numbers of SMTP providers don't verify anything beyond the login. As a result, each and every one who manages to get an account can send legitimate looking messages for whatever domain allowing that host, e.g. another tenant. Microsoft and Google do perform sender address checks, as well as TransIP that I know of. Apart from them, though... For reference: in Postfix this can be configured with smtpd_sender_restrictions.
charamis 3y ago
I’m trying to setup my own domain for my email address since yesterday and this really hit the spot.
- zenorocha 3y ago
  So happy to see this was useful!
- jeroenhd 3y ago
  One additional (non-security) feature that this guide missed is a reverse PTR record for your server's IP address. You'll have to set this on the side where you get your IP address from rather than in your own DNS, but lacking it will get your email classified as spam even if you implement all the other protocols. Basically, when your mail server says "EHLO mail.chamaris.tld", the other side will try to validate that the IP address you're connecting from actually points to mail.chamaris.tld. Furthermore, you need to pick a host that takes action when their IP addresses appear on blacklists. Microsoft, particularly, uses UCEPROTECT as a blacklist source, and that particular company will blacklist and entire ASN if they receive too much spam from a particular IP address block (that doesn't get resolved in time).
  - Avamander 3y ago
    > Microsoft, particularly, uses UCEPROTECT as a blacklist source, and that particular company will blacklist and entire ASN if they receive too much spam from a particular IP address block (that doesn't get resolved in time) They absolutely do not use that crap.
  - charamis 3y ago
    Glad you're pointing it out. I still have problems, indeed, having my emails flagged as spam when sent to outlook.com. On gmail.com they go straight to the inbox. If I understand correctly though, I should have to set a reverse PTR record in the case of setting up my own mail server, right? And I should configure it on the mailserver's side, correct? That's not my case though, I'm just using a custom domain with the MX records pointing to the mail service that I'm using. I'll just wait a few days cause the domain is new and I guess that's the reason outlook.com flags it.
upofadown 3y ago
This is new: >BIMI (Brand Indicators for Message Identification) is this kind of access in the inbox. It sets you apart from all the others by showcasing your brand and legitimacy to your users in the inbox by displaying your logo and, in some cases, a verified checkmark. Apparently this can help me promote my brand. Unfortunately I don't have a brand so I fear that this would be used to promote the brands of others at my expense.
hashstring 3y ago
Anyone willing to share opinions on BIMI? I’m wondering if it is worth it for most medium-large organization or if this is specifically worth it if you are doing a lot of commerce and sending e-mails to customers etc. Furthermore, (stating the obvious) DKIM, SPF and DMARC are also implemented by malicious parties and only authenticate that the server was allowed to send using a particular domain name. BIMI seems to require a VMC (Verified Mark Certificate). Is this verified and is it effective in preventing unauthorized parties from BIMI verifying their domains using stolen brand logo’s etc. Also, is Microsoft Outlook (still) not supporting/adopting BIMI?
- layer8 3y ago
  As an end user, this is gimmicky to me, and I wouldn’t want to use an email client that causes the respective emails to appear more prominent by showing their BIMI logo. It would be similarly annoying as emojis in the subject line.
  - Avamander 3y ago
    A lot of mail clients do display it though. Plus it does help the average user differentiate between rnicrosoft.com and microsoft.com, I'd guess.
- brightball 3y ago
  Happy to. BIMI is worthless. It's a carrot to get your marketing department on board with setting up DMARC because they are the most likely to push back due to fears of the project affecting their email deliverability. You have to fully deploy DMARC to setup BIMI. Now, BIMI and the costs to "verify" your mark is pushed by exactly the same people who tried to sell you extended validation SSL that would turn your browser address bar green when you visited a site that had gone through this verification. Just like with EV SSL, BIMI has no positive impact on user security. They're just as likely to open / not open an email with BIMI as they are to visit a site with or without EV SSL. In some cases, it's actually worse. The only benefit to BIMI is it gives you another place for the marketing department to stick the logo so they'll stop fighting the DMARC rollout. That's it. Otherwise it's total waste of money and time. Wrote about it here: https://www.brightball.com/articles/enterprise-challenges-with-dmarc-deployment https://www.brightball.com/articles/enterprise-challenges-wi...
Systemmanic 3y ago
I liked the look of this: >BIMI (Brand Indicators for Message Identification) is this kind of access in the inbox. It sets you apart from all the others by showcasing your brand and legitimacy to your users in the inbox by displaying your logo and, in some cases, a verified checkmark. Until I looked at the cost of a Verified Mark Certificate (1-year plan): >$1,499.00 USD [1] Yikes. Small money for big players, but small businesses with valid brands not so much. [1]https://order.digicert.com/step1/vmc_basic https://order.digicert.com/step1/vmc_basic
- donmcronald 3y ago
  $1500 USD / year for something that collides with the logo functionality of Gravatar if the user isn't hovering over the logo? We used to emphasize domains and everyone understood them. Then the large tech companies de-emphasized domains to the point where people stopped understanding them. Now big tech is going to sell domain validation back to us at a premium? Wow! What innovation. I know there's trademark verification too, but I've never met a normal person that could tell you a difference between a Gravatar logo like the one I see in my mail client and a VMC logo like the one I see in the screenshots, so what good is showing a trademarked logo? Also, most small businesses I've seen don't even have trademarks, so they'll be completely excluded from this system. I wonder if this is going to turn out like code signing certificates where they're super expensive for small developers, so they get excluded, but they're totally attainable for scammers and scumbags, so there's plenty of malware and garbage signed by certificates from fly-by-night companies. Does BIMI help you pass spam filters like EV code signing certificates help you bypass SmartScreen? I can't be the only one that thinks all these things feel like a scam. One thing I'm certain of based on what we see with SSL certificates. Government agencies will be racing to light money on fire buying them. Every year I watch my taxes get spent on overpriced DigiCert OV certificates and it enrages me. For all intents and purposes, all certificates are identical to normal users. It doesn't matter if DigiCert is taking my DNA for validation, all my mom sees is the lock icon. Nothing else matters.
  - Avamander 3y ago
    > I know there's trademark verification too, but I've never met a normal person that could tell you a difference between a Gravatar logo Current implementations display a blue checkmark in addition to the logo. It's a bit different from what Gmail or Gravatar previously has done. > Does BIMI help you pass spam filters like EV code signing certificates help you bypass SmartScreen? I can't be the only one that thinks all these things feel like a scam. Having a proper SPF/DKIM/DMARC setup most likely has the biggest impact, but BIMI might also be taken into account.
stuckinaloop 3y ago
Hey Resend team! Love your product! I was wondering when / if you'll be supporting inbound email parsing / webhooks. It's the one thing that's preventing me from switching over. Love what you're doing - keep it up!
- zenorocha 3y ago
  Inbound emails is definitely one of the most frequent feature requests we get. We already started to investigate it and I hope we can ship something in the next couple months.
  - stuckinaloop 3y ago
    Awesome! Will keep my eyes peeled
deleted 3y ago
[deleted]
arcade79 3y ago
I'm trying to understand why all BIMI-mail shouldn't be simply rejected on the SMTP layer ..
csharpminor 3y ago
To be honest I found this article to be a bit blog spammy. It’s for developers because they used a startup accelerator analogy? I didn’t see anything particularly differentiated from the equivalent SendGrid/Mailgun/etc blogs that come up. This is probably the best demo for understanding SPF/DKIM/DMARC that I’ve come across (I am not affiliated): https://www.learndmarc.com/ https://www.learndmarc.com/ If you really want to understand DMARC check it out.
- jmholla 3y ago
  I heavily agree. The analogy was distracting at best, confusing at worst. Just explaining what each feature does is perfectly grokkable. We all know what email is.
- freddieleeman 3y ago
  Totally agree, this blog makes no sense at all. "If a server isn't on the list, it's like an application being tossed out because it wasn't fully filled or the business idea is illegal." Seriously, what?! "DMARC establishes your policy as a sender for what should happen to your messages if they fail DKIM or SPF." ... only if they fail DKIM AND SPF (and alignment). The whole blog doesn't even talk about identifier alignment, which is a big part of DMARC. Maybe an AI wrote the article and got stuck trying to use a weird analogy about a "competitive accelerator program" for email security? :)
thereald0tt 3y ago
Can someone tell me what is it that resend provides over their wrapper (AWS) that makes them this famous?
3dbrows 3y ago
Does Resend use AWS SES under the hood?
- pembrook 3y ago
  From what I can tell, yes.

209 points by zenorocha 3y ago

Note that this is about the use case 'authenticating whether the given mail server is authorized to send mail for the given domain', not the use cases 'authenticating whether the given user of this mailbox actually sent this email' or 'authenticating whether the given message is spam or not', which is probably the reason you clicked on this article :). The former use case is pretty much solved (in that you can safely ignore email from servers/domains that don't follow best practices), the latter (combining the 2, since they're pretty similar, really) is not, even given recent advances in AI (OpenAI cannot tell you if a message is spam, sorry, unless your prompt engineering skills are much better than mine).

hashstring 3y ago
> OpenAI cannot tell you if a message is spam, sorry, unless your prompt engineering skills are much better than mine. Sometimes an e-mail message itself does not even contain enough information to accurately classify it as spam or phish. To a degree, spam is subjective. And classifying a phish may not be trivial at all (e.g. message may include legit marketing links, open redirects and server side logic to serve certain pages only to targets, etc.).
- bruce343434 3y ago
  Yup, when I order something I get really annoyed that I get an email for every fart that the delivery driver lets out. I'll hear the doorbell, I don't need 500 anticipatory emails. It's not a scam, not a phish, and it's 100% factual and "informative". But still junk.
  - akho 3y ago
    It isn't spam though (being a part of a real commercial relationship, and having a working unsubscribe link), and marking it as such poisons your spam filter.
- heipei 3y ago
  Thank you for that comment, I don't think it's something many people really understood. The same is true for phishing websites. So much depends on the context and incomplete background information. Is a website that asks you to put in your username and password bad? Well, it depends on what the website does with that information in the background. I've seen very suspicious websites asking for user information which were, in the end, just sites set up by marketing departments of the larger company who were unaware of the dangerous precedent they were setting.
thijsvandien 3y ago
To further emphasize the point that a mail server being authorized often isn't worth much, or can be counterproductive even: shocking numbers of SMTP providers don't verify anything beyond the login. As a result, each and every one who manages to get an account can send legitimate looking messages for whatever domain allowing that host, e.g. another tenant. Microsoft and Google do perform sender address checks, as well as TransIP that I know of. Apart from them, though... For reference: in Postfix this can be configured with smtpd_sender_restrictions.

charamis 3y ago

I’m trying to setup my own domain for my email address since yesterday and this really hit the spot.

zenorocha 3y ago
So happy to see this was useful!
jeroenhd 3y ago
One additional (non-security) feature that this guide missed is a reverse PTR record for your server's IP address. You'll have to set this on the side where you get your IP address from rather than in your own DNS, but lacking it will get your email classified as spam even if you implement all the other protocols. Basically, when your mail server says "EHLO mail.chamaris.tld", the other side will try to validate that the IP address you're connecting from actually points to mail.chamaris.tld. Furthermore, you need to pick a host that takes action when their IP addresses appear on blacklists. Microsoft, particularly, uses UCEPROTECT as a blacklist source, and that particular company will blacklist and entire ASN if they receive too much spam from a particular IP address block (that doesn't get resolved in time).
- Avamander 3y ago
  > Microsoft, particularly, uses UCEPROTECT as a blacklist source, and that particular company will blacklist and entire ASN if they receive too much spam from a particular IP address block (that doesn't get resolved in time) They absolutely do not use that crap.
- charamis 3y ago
  Glad you're pointing it out. I still have problems, indeed, having my emails flagged as spam when sent to outlook.com. On gmail.com they go straight to the inbox. If I understand correctly though, I should have to set a reverse PTR record in the case of setting up my own mail server, right? And I should configure it on the mailserver's side, correct? That's not my case though, I'm just using a custom domain with the MX records pointing to the mail service that I'm using. I'll just wait a few days cause the domain is new and I guess that's the reason outlook.com flags it.

upofadown 3y ago

This is new: >BIMI (Brand Indicators for Message Identification) is this kind of access in the inbox. It sets you apart from all the others by showcasing your brand and legitimacy to your users in the inbox by displaying your logo and, in some cases, a verified checkmark. Apparently this can help me promote my brand. Unfortunately I don't have a brand so I fear that this would be used to promote the brands of others at my expense.

hashstring 3y ago

Anyone willing to share opinions on BIMI? I’m wondering if it is worth it for most medium-large organization or if this is specifically worth it if you are doing a lot of commerce and sending e-mails to customers etc. Furthermore, (stating the obvious) DKIM, SPF and DMARC are also implemented by malicious parties and only authenticate that the server was allowed to send using a particular domain name. BIMI seems to require a VMC (Verified Mark Certificate). Is this verified and is it effective in preventing unauthorized parties from BIMI verifying their domains using stolen brand logo’s etc. Also, is Microsoft Outlook (still) not supporting/adopting BIMI?

layer8 3y ago
As an end user, this is gimmicky to me, and I wouldn’t want to use an email client that causes the respective emails to appear more prominent by showing their BIMI logo. It would be similarly annoying as emojis in the subject line.
- Avamander 3y ago
  A lot of mail clients do display it though. Plus it does help the average user differentiate between rnicrosoft.com and microsoft.com, I'd guess.
brightball 3y ago
Happy to. BIMI is worthless. It's a carrot to get your marketing department on board with setting up DMARC because they are the most likely to push back due to fears of the project affecting their email deliverability. You have to fully deploy DMARC to setup BIMI. Now, BIMI and the costs to "verify" your mark is pushed by exactly the same people who tried to sell you extended validation SSL that would turn your browser address bar green when you visited a site that had gone through this verification. Just like with EV SSL, BIMI has no positive impact on user security. They're just as likely to open / not open an email with BIMI as they are to visit a site with or without EV SSL. In some cases, it's actually worse. The only benefit to BIMI is it gives you another place for the marketing department to stick the logo so they'll stop fighting the DMARC rollout. That's it. Otherwise it's total waste of money and time. Wrote about it here: https://www.brightball.com/articles/enterprise-challenges-with-dmarc-deployment https://www.brightball.com/articles/enterprise-challenges-wi...

Systemmanic 3y ago

I liked the look of this: >BIMI (Brand Indicators for Message Identification) is this kind of access in the inbox. It sets you apart from all the others by showcasing your brand and legitimacy to your users in the inbox by displaying your logo and, in some cases, a verified checkmark. Until I looked at the cost of a Verified Mark Certificate (1-year plan): >$1,499.00 USD [1] Yikes. Small money for big players, but small businesses with valid brands not so much. [1]https://order.digicert.com/step1/vmc_basic https://order.digicert.com/step1/vmc_basic

donmcronald 3y ago
$1500 USD / year for something that collides with the logo functionality of Gravatar if the user isn't hovering over the logo? We used to emphasize domains and everyone understood them. Then the large tech companies de-emphasized domains to the point where people stopped understanding them. Now big tech is going to sell domain validation back to us at a premium? Wow! What innovation. I know there's trademark verification too, but I've never met a normal person that could tell you a difference between a Gravatar logo like the one I see in my mail client and a VMC logo like the one I see in the screenshots, so what good is showing a trademarked logo? Also, most small businesses I've seen don't even have trademarks, so they'll be completely excluded from this system. I wonder if this is going to turn out like code signing certificates where they're super expensive for small developers, so they get excluded, but they're totally attainable for scammers and scumbags, so there's plenty of malware and garbage signed by certificates from fly-by-night companies. Does BIMI help you pass spam filters like EV code signing certificates help you bypass SmartScreen? I can't be the only one that thinks all these things feel like a scam. One thing I'm certain of based on what we see with SSL certificates. Government agencies will be racing to light money on fire buying them. Every year I watch my taxes get spent on overpriced DigiCert OV certificates and it enrages me. For all intents and purposes, all certificates are identical to normal users. It doesn't matter if DigiCert is taking my DNA for validation, all my mom sees is the lock icon. Nothing else matters.
- Avamander 3y ago
  > I know there's trademark verification too, but I've never met a normal person that could tell you a difference between a Gravatar logo Current implementations display a blue checkmark in addition to the logo. It's a bit different from what Gmail or Gravatar previously has done. > Does BIMI help you pass spam filters like EV code signing certificates help you bypass SmartScreen? I can't be the only one that thinks all these things feel like a scam. Having a proper SPF/DKIM/DMARC setup most likely has the biggest impact, but BIMI might also be taken into account.

stuckinaloop 3y ago

Hey Resend team! Love your product! I was wondering when / if you'll be supporting inbound email parsing / webhooks. It's the one thing that's preventing me from switching over. Love what you're doing - keep it up!

zenorocha 3y ago
Inbound emails is definitely one of the most frequent feature requests we get. We already started to investigate it and I hope we can ship something in the next couple months.
- stuckinaloop 3y ago
  Awesome! Will keep my eyes peeled

deleted 3y ago

[deleted]

arcade79 3y ago

I'm trying to understand why all BIMI-mail shouldn't be simply rejected on the SMTP layer ..

csharpminor 3y ago

To be honest I found this article to be a bit blog spammy. It’s for developers because they used a startup accelerator analogy? I didn’t see anything particularly differentiated from the equivalent SendGrid/Mailgun/etc blogs that come up. This is probably the best demo for understanding SPF/DKIM/DMARC that I’ve come across (I am not affiliated): https://www.learndmarc.com/ https://www.learndmarc.com/ If you really want to understand DMARC check it out.

jmholla 3y ago
I heavily agree. The analogy was distracting at best, confusing at worst. Just explaining what each feature does is perfectly grokkable. We all know what email is.
freddieleeman 3y ago
Totally agree, this blog makes no sense at all. "If a server isn't on the list, it's like an application being tossed out because it wasn't fully filled or the business idea is illegal." Seriously, what?! "DMARC establishes your policy as a sender for what should happen to your messages if they fail DKIM or SPF." ... only if they fail DKIM AND SPF (and alignment). The whole blog doesn't even talk about identifier alignment, which is a big part of DMARC. Maybe an AI wrote the article and got stuck trying to use a weird analogy about a "competitive accelerator program" for email security? :)

thereald0tt 3y ago

Can someone tell me what is it that resend provides over their wrapper (AWS) that makes them this famous?

3dbrows 3y ago

Does Resend use AWS SES under the hood?

pembrook 3y ago
From what I can tell, yes.