4 ms·

Gmail randomly(?) requires “new senders” to have SPF records since November '22

13 points by alturiak 3y ago

alturiak 3y ago
This doesn't seem to have hit any major tech news outlets at the time, but since I've stumbled upon it twice in a short timeframe, I can only assume that others have been bitten by it: "Starting November 2022, new senders who send email to personal Gmail accounts must set up either SPF or DKIM. Google performs random checks on new sender messages to personal Gmail accounts to verify they’re authenticated." There is no explanation on how the randomness in this works, but I can't imagine it would be helpful in troubleshooting related issues.
- LeonM 3y ago
  There is no randomness, this is how DMARC works. You need SPF or DKIM alignment (either one will do). Basically what is happening, is that Gmail is slowly enforcing DMARC for new domains. This is a good thing, any domain that sends email should use DMARC, and all email should be properly signed with a DKIM signature. Adopting DMARC can be difficult for existing domains, but if implemented from day one (what Google is mandating here for 'new' domains) it is fairly trivial. That all said: if one is setting up email for any domain, you should focus on DKIM, rather than SPF. SPF is horribly flawed in many ways, and can break in many ways. If you ensure that all your email is DKIM signed (and aligned), you'll have covered 99% of deliverabilty. You should still enable SPF though to support the 1% of legacy email services that don't support DKIM, but just don't rely on SPF alone.
  - alturiak 3y ago
    Thanks for the elaboration that certainly adds useful context. While I personally am aware of DMARC and haven't been hit by this, I've heard from friends that small businesses aren't able to contact them on their gmail-address anymore due to them (the businesses) not having setup neither SPF nor DKIM. I fully agree that this shouldn't be the case anymore in this day and age, yet here we are. The "random"-bit is directly mentioned in Google's support article and I can't figure out what it means. Either they enforce it for a sender's domain or they don't. My best guess is that they randomly decide when to start enforcing it for new or (probably more likely at this point) previously known individual domains.
- jseliger 3y ago
  I got hit by this with seliger.com domain emails, although seliger.com predates gmail.
lakomen 3y ago
Yeah I've been hit by this on multiple domains. It hindered my Keycloak registration confirmation mails from Google Oauth/oidc federated signin, leading to having to manually enable those accounts. If your MTA is on the same IP/host as your A and/or AAAA record/s, the following TXT record solves that: "v=spf1 mx a -all" If you don't have a MX record remove the mx part. EZ This essentially means SPF version 1, Trust the MX IP, Trust all the IPs of that domain, Reject all other IPs.
- LeonM 3y ago
  Be careful with doing this, be sure you understand the implications of simply copy/pasting such an SPF record. For most situations, your MX (which is inbound email) may not necessarily be sending (outbound) email. For example: when using O365 or Google Workspace (Gmail), then these are always separated. You need to use the 'include' mechanism for your email inbox providers, and not the 'mx' mechanism. Leaving the 'mx' mechanism in your SPF when it is not required is rather expensive as it causes an additional DNS lookup, and you can only have 10 additional lookups in an SPF policy. The 'a' is also an additional lookup, and may not always be applicable. The host pointed to by the 'a' record may not (and probably should not) be sending email. In parent's example there is a Keycloak server running on the host pointed to by the 'a' record. It would better configure Keycloak to user a 'real' SMTP server, instead of attempting to deliver the confirmation emails directly from your host. Also, you really should enable DKIM signing for those emails, never rely on SPF for such critical emails. Lastly, using the '-all' default can be dangerous when you are just setting up SPF for the first time. You may end up blocking a lot of your own (legitimate) email. When SPF is combined with DMARC, it is even recommended to never use '-all' at all. https://www.mailhardener.com/blog/why-mailhardener-recommends-spf-softfail-over-fail https://www.mailhardener.com/blog/why-mailhardener-recommend... TL;DR: Don't just blatantly copy/paste an SPF example, be sure to understand what you are doing, or you may end up hurting your email deliverability. There are many good resources online on the subject, such as https://www.mailhardener.com/kb/spf https://www.mailhardener.com/kb/spf (disclaimer: I work at Mailhardener)

13 points by alturiak 3y ago

This doesn't seem to have hit any major tech news outlets at the time, but since I've stumbled upon it twice in a short timeframe, I can only assume that others have been bitten by it: "Starting November 2022, new senders who send email to personal Gmail accounts must set up either SPF or DKIM. Google performs random checks on new sender messages to personal Gmail accounts to verify they’re authenticated." There is no explanation on how the randomness in this works, but I can't imagine it would be helpful in troubleshooting related issues.

LeonM 3y ago
There is no randomness, this is how DMARC works. You need SPF or DKIM alignment (either one will do). Basically what is happening, is that Gmail is slowly enforcing DMARC for new domains. This is a good thing, any domain that sends email should use DMARC, and all email should be properly signed with a DKIM signature. Adopting DMARC can be difficult for existing domains, but if implemented from day one (what Google is mandating here for 'new' domains) it is fairly trivial. That all said: if one is setting up email for any domain, you should focus on DKIM, rather than SPF. SPF is horribly flawed in many ways, and can break in many ways. If you ensure that all your email is DKIM signed (and aligned), you'll have covered 99% of deliverabilty. You should still enable SPF though to support the 1% of legacy email services that don't support DKIM, but just don't rely on SPF alone.
- alturiak 3y ago
  Thanks for the elaboration that certainly adds useful context. While I personally am aware of DMARC and haven't been hit by this, I've heard from friends that small businesses aren't able to contact them on their gmail-address anymore due to them (the businesses) not having setup neither SPF nor DKIM. I fully agree that this shouldn't be the case anymore in this day and age, yet here we are. The "random"-bit is directly mentioned in Google's support article and I can't figure out what it means. Either they enforce it for a sender's domain or they don't. My best guess is that they randomly decide when to start enforcing it for new or (probably more likely at this point) previously known individual domains.
jseliger 3y ago
I got hit by this with seliger.com domain emails, although seliger.com predates gmail.

lakomen 3y ago

Yeah I've been hit by this on multiple domains. It hindered my Keycloak registration confirmation mails from Google Oauth/oidc federated signin, leading to having to manually enable those accounts. If your MTA is on the same IP/host as your A and/or AAAA record/s, the following TXT record solves that: "v=spf1 mx a -all" If you don't have a MX record remove the mx part. EZ This essentially means SPF version 1, Trust the MX IP, Trust all the IPs of that domain, Reject all other IPs.

LeonM 3y ago
Be careful with doing this, be sure you understand the implications of simply copy/pasting such an SPF record. For most situations, your MX (which is inbound email) may not necessarily be sending (outbound) email. For example: when using O365 or Google Workspace (Gmail), then these are always separated. You need to use the 'include' mechanism for your email inbox providers, and not the 'mx' mechanism. Leaving the 'mx' mechanism in your SPF when it is not required is rather expensive as it causes an additional DNS lookup, and you can only have 10 additional lookups in an SPF policy. The 'a' is also an additional lookup, and may not always be applicable. The host pointed to by the 'a' record may not (and probably should not) be sending email. In parent's example there is a Keycloak server running on the host pointed to by the 'a' record. It would better configure Keycloak to user a 'real' SMTP server, instead of attempting to deliver the confirmation emails directly from your host. Also, you really should enable DKIM signing for those emails, never rely on SPF for such critical emails. Lastly, using the '-all' default can be dangerous when you are just setting up SPF for the first time. You may end up blocking a lot of your own (legitimate) email. When SPF is combined with DMARC, it is even recommended to never use '-all' at all. https://www.mailhardener.com/blog/why-mailhardener-recommends-spf-softfail-over-fail https://www.mailhardener.com/blog/why-mailhardener-recommend... TL;DR: Don't just blatantly copy/paste an SPF example, be sure to understand what you are doing, or you may end up hurting your email deliverability. There are many good resources online on the subject, such as https://www.mailhardener.com/kb/spf https://www.mailhardener.com/kb/spf (disclaimer: I work at Mailhardener)