5 ms·

Sending mail with AWS SES and Route53

116 points by chrisanthropic 10y ago

chrisanthropic 10y ago
And just in case anyone is feeling frisky all of the sensitive info in the screenshots was removed and replaced.
inopinatus 10y ago
If you just enable DMARC like that you'll receive a XML report every day from every major provider. To reduce the noise and do something useful with this data consider using an aggregator tool e.g. https://dmarc.postmarkapp.com https://dmarc.postmarkapp.com (free and from a trustworthy source)
- chrisanthropic 10y ago
  Great info, thanks, setting up my account now and then I'll update the article. EDIT The article has been updated to include this.
- icebraining 10y ago
  +1, we use it despite not being users of Postmark itself (we just don't send many transactional emails), and it's great. Very clear and reliable.
- homero 10y ago
  I use https://dmarcian.com https://dmarcian.com
- technion 10y ago
  Does anyone really do anything useful with said reports though? In my experience... In the first week you identify a web server or two some marketing team put together with a contact form that you never realised spoofed your domain and never got past SPF filters. After that, you get hundreds of alerts a day about some server in China sending spoofed emails to a server in Vietnam, neither of which you can do anything about.
  - inopinatus 10y ago
    That is why you should throw the reports at an aggregator. I'm very happy with Postmark's weekly digest.
chao- 10y ago
There are a solid handful of these "newsletter/marketing over SES" services these days. Sendy, Moon Mail, EmailOctopus and a few others (last did a large search a few months ago). Sendy even has a value-add service piggybacking off them (EasySendy Pro), though it looks like they've significantly added features since the last time I looked. But where most people tend to see "Amazon == No Need To Think", I also see a looming "all your eggs in someone else's basket" and the first thing I look for is whether I can plug in an SMTP provider that isn't Amazon SES. In the FAQ, in a sub-note on a Features page, or anywhere. Almost without fail, none of their sites address this, even though it seems like it wouldn't be much harder than being SES specific. Perhaps spam reports and bounce tracking might be sacrificed (i.e. requires outsized effort to implement) if it was via generic SMTP and not SES specific? Does anyone know the answer to the "any SMTP" question for any of these services?
- chrisanthropic 10y ago
  I've spent the last few months researching this and the only one I've found that offers that is https://mailtrain.org/ https://mailtrain.org/ It's self-hosted and from their FAQ: "Amazon SES, SparkPost, SendGrid, Mailgun – you name it. You can use any provider that supports SMTP protocol to send out your newsletters. Bounce and complaints handling via webhooks is supported for SES, SparkPost, SendGrid and Mailgun."
  - chao- 10y ago
    I remember finding Mailtrain in my search, but the last time I looked, I don't remember seeing some of the details that I'm now reading on their Github. Apparently they can read bounces from more than just the services you named (which itself is a good improvement over Amazon-only), but can read from ZoneMTA and Postfix logs as well which is awesome and solves my "someone else's basket" concerns. Thanks for prompting me to take a look at them again!
    - chrisanthropic 10y ago
      No problem. Honestly, the only reason I didn't go with that is because I work with AWS and wanted to see how they're setting up a 'serverless' newsletter via SES.
vacri 10y ago
An issue that recently hit us: if you're going to be testing mail, then use the SES test addresses. Don't use your own or fake addresses. If SES get x% bounces on any outbound address, it will cut off your SES access across the board, and it doesn't come back on quickly. We had a test address that was purposefully undeliverable. A test script sent out thousands of mails when it shouldn't have, and those undeliverable mails got treated as bounces. So, we got our SES cut off for two days, despite our clearly test/undeliverable mail being the cause. Regular AWS support can't do anything, only a special email unlock team can (they protect the 'deliverability' of AWS mail), and they're not exactly responsive. http://docs.aws.amazon.com/ses/latest/DeveloperGuide/mailbox-simulator.html http://docs.aws.amazon.com/ses/latest/DeveloperGuide/mailbox...
- koko775 10y ago
  >_< Yeah. Bounces are bounces, and the mail reputation score probably won't care that you're testing things when it penalizes Amazon for its IPs sending bogus mail.
  - vacri 10y ago
    To clarify, I meant undeliverable as in "domain is not in DNS", not "mail is rejected by recipient", sorry. There was nowhere for the mail to be delivered to.
- somecallitblues 10y ago
  Always use mailinator.com
justinator 10y ago
Dada Mail [0] (which I wrote, and have worked on for 15+ years) supports sending via SES, and it works great. The instructions to implement it [1] are quite similar. I set up SES for a ton of clients, and the majority of them seem very happy using it. One thing this article doesn't touch upon is that SES does have a limit on how many messages you may send in a timeframe per second, as well as per day. If you go over these limits, your message will not send out correctly. Make sure your software supports enough of the API for SES to fetch these limits are correctly send your messages, below these limits. [0] http://dadamailproject.com/ http://dadamailproject.com/ [1] http://dadamailproject.com/d/features-amazon_ses_support.pod.html http://dadamailproject.com/d/features-amazon_ses_support.pod...
questionr 10y ago
whats the Google Cloud (first-party) managed mail service for personal domains?
kennysmoothx 10y ago
I have moved over all of my clients who use transactional email over to SES from Sendgrid and honestly they couldn't be happier. SES Pricing is amazing and deliverability seems to be good all across. (Not to mention you get around 60k free emails monthly if requests are coming from a AWS server)
SnowingXIV 10y ago
What do people do these days for easily handling contact forms with a shared host? They come from my server name which then I change, so sometimes could be marked as spam. Simple php post. To/From
- JoshTriplett 10y ago
  Always send those mails from a specifically configured real email address at your application's domain, not involving the name of your server. For instance, send the mail "From: contact-form@example.com", and "To: wherever-that-mail-goes@example.com". And send the mail using the actual mail server that handles @example.com mail (ideally not one you administrate). Don't include any user-supplied email address in either From or To; that'll get your mails either marked as spam or used as a vehicle for actual spam.
- rokhayakebe 10y ago
  Mailgun.
  - chrisanthropic 10y ago
    This.
  - somecallitblues 10y ago
    Did this exist when Mandrill decided to boot their customers? I migrated all my clients that were on it to sendgrid but this would have been a good option for the cheap ones
  - somecallitblues 10y ago
    Did this exist when Mandrill decided to boot their customers? I migrated all my clients that were on it to sendgrid but this would have been a good option for the cheap ones
wineisfine 10y ago
I tried (bought) sendy.co and it seems quite buggy. I'm still looking for something self-hosted, where any SMTP provider can be plugged into, and that does not look like its 1993. As I have some non-techies that need to work with it.
- chrisanthropic 10y ago
  Like I said further down, give mailtrain.org a look. It's self hosted and you can use multiple SMTP providers.
  - nickjj 10y ago
    The problem with mailtrain is according to them it's not viable to use in production. There's also no tests and very little documentation. I can get around that but the main thing preventing me from using it is there's no success stories associated to the application. Sendy has dozens of unbiased blog posts, and even some posts where people are sending millions of e-mails without issues. That instills a lot of confidence in using it. You would have to be a madman to trust an untrusted application with managing your email subscribers.
    - andris9 10y ago
      Just for the record, Mailtrain is used in production by several marketers with very large lists (from hundred thousand subscribers to a million subscribers) and Mailtrain seems to work fine for them
      - nickjj 10y ago
        Is there anything to read that's public where those marketers comment about using it? Have any of them done any audits on the code base and traced all incoming requests through Mailtrain vs their SES backend to ensure everything was delivered as expected?
- slig 10y ago
  Try Mailwizz. It has tons of features and the author is very active on the support forum.
jonathanbull 10y ago
This is really useful - DKIM/SPF/DMARC setup is way trickier than it should be. I'm a co-founder of EmailOctopus so happy to answer any questions on the integration side of things.

116 points by chrisanthropic 10y ago

And just in case anyone is feeling frisky all of the sensitive info in the screenshots was removed and replaced.

If you just enable DMARC like that you'll receive a XML report every day from every major provider. To reduce the noise and do something useful with this data consider using an aggregator tool e.g. https://dmarc.postmarkapp.com https://dmarc.postmarkapp.com (free and from a trustworthy source)

chrisanthropic 10y ago
Great info, thanks, setting up my account now and then I'll update the article. EDIT The article has been updated to include this.
icebraining 10y ago
+1, we use it despite not being users of Postmark itself (we just don't send many transactional emails), and it's great. Very clear and reliable.
homero 10y ago
I use https://dmarcian.com https://dmarcian.com
technion 10y ago
Does anyone really do anything useful with said reports though? In my experience... In the first week you identify a web server or two some marketing team put together with a contact form that you never realised spoofed your domain and never got past SPF filters. After that, you get hundreds of alerts a day about some server in China sending spoofed emails to a server in Vietnam, neither of which you can do anything about.
- inopinatus 10y ago
  That is why you should throw the reports at an aggregator. I'm very happy with Postmark's weekly digest.

chao- 10y ago

There are a solid handful of these "newsletter/marketing over SES" services these days. Sendy, Moon Mail, EmailOctopus and a few others (last did a large search a few months ago). Sendy even has a value-add service piggybacking off them (EasySendy Pro), though it looks like they've significantly added features since the last time I looked. But where most people tend to see "Amazon == No Need To Think", I also see a looming "all your eggs in someone else's basket" and the first thing I look for is whether I can plug in an SMTP provider that isn't Amazon SES. In the FAQ, in a sub-note on a Features page, or anywhere. Almost without fail, none of their sites address this, even though it seems like it wouldn't be much harder than being SES specific. Perhaps spam reports and bounce tracking might be sacrificed (i.e. requires outsized effort to implement) if it was via generic SMTP and not SES specific? Does anyone know the answer to the "any SMTP" question for any of these services?

chrisanthropic 10y ago
I've spent the last few months researching this and the only one I've found that offers that is https://mailtrain.org/ https://mailtrain.org/ It's self-hosted and from their FAQ: "Amazon SES, SparkPost, SendGrid, Mailgun – you name it. You can use any provider that supports SMTP protocol to send out your newsletters. Bounce and complaints handling via webhooks is supported for SES, SparkPost, SendGrid and Mailgun."
- chao- 10y ago
  I remember finding Mailtrain in my search, but the last time I looked, I don't remember seeing some of the details that I'm now reading on their Github. Apparently they can read bounces from more than just the services you named (which itself is a good improvement over Amazon-only), but can read from ZoneMTA and Postfix logs as well which is awesome and solves my "someone else's basket" concerns. Thanks for prompting me to take a look at them again!
  - chrisanthropic 10y ago
    No problem. Honestly, the only reason I didn't go with that is because I work with AWS and wanted to see how they're setting up a 'serverless' newsletter via SES.

vacri 10y ago

An issue that recently hit us: if you're going to be testing mail, then use the SES test addresses. Don't use your own or fake addresses. If SES get x% bounces on any outbound address, it will cut off your SES access across the board, and it doesn't come back on quickly. We had a test address that was purposefully undeliverable. A test script sent out thousands of mails when it shouldn't have, and those undeliverable mails got treated as bounces. So, we got our SES cut off for two days, despite our clearly test/undeliverable mail being the cause. Regular AWS support can't do anything, only a special email unlock team can (they protect the 'deliverability' of AWS mail), and they're not exactly responsive. http://docs.aws.amazon.com/ses/latest/DeveloperGuide/mailbox-simulator.html http://docs.aws.amazon.com/ses/latest/DeveloperGuide/mailbox...

koko775 10y ago
>_< Yeah. Bounces are bounces, and the mail reputation score probably won't care that you're testing things when it penalizes Amazon for its IPs sending bogus mail.
- vacri 10y ago
  To clarify, I meant undeliverable as in "domain is not in DNS", not "mail is rejected by recipient", sorry. There was nowhere for the mail to be delivered to.
somecallitblues 10y ago
Always use mailinator.com

justinator 10y ago

Dada Mail [0] (which I wrote, and have worked on for 15+ years) supports sending via SES, and it works great. The instructions to implement it [1] are quite similar. I set up SES for a ton of clients, and the majority of them seem very happy using it. One thing this article doesn't touch upon is that SES does have a limit on how many messages you may send in a timeframe per second, as well as per day. If you go over these limits, your message will not send out correctly. Make sure your software supports enough of the API for SES to fetch these limits are correctly send your messages, below these limits. [0] http://dadamailproject.com/ http://dadamailproject.com/ [1] http://dadamailproject.com/d/features-amazon_ses_support.pod.html http://dadamailproject.com/d/features-amazon_ses_support.pod...

questionr 10y ago

whats the Google Cloud (first-party) managed mail service for personal domains?

kennysmoothx 10y ago

I have moved over all of my clients who use transactional email over to SES from Sendgrid and honestly they couldn't be happier. SES Pricing is amazing and deliverability seems to be good all across. (Not to mention you get around 60k free emails monthly if requests are coming from a AWS server)

SnowingXIV 10y ago

What do people do these days for easily handling contact forms with a shared host? They come from my server name which then I change, so sometimes could be marked as spam. Simple php post. To/From

JoshTriplett 10y ago
Always send those mails from a specifically configured real email address at your application's domain, not involving the name of your server. For instance, send the mail "From: contact-form@example.com", and "To: wherever-that-mail-goes@example.com". And send the mail using the actual mail server that handles @example.com mail (ideally not one you administrate). Don't include any user-supplied email address in either From or To; that'll get your mails either marked as spam or used as a vehicle for actual spam.
rokhayakebe 10y ago
Mailgun.
- chrisanthropic 10y ago
  This.
- somecallitblues 10y ago
  Did this exist when Mandrill decided to boot their customers? I migrated all my clients that were on it to sendgrid but this would have been a good option for the cheap ones
- somecallitblues 10y ago
  Did this exist when Mandrill decided to boot their customers? I migrated all my clients that were on it to sendgrid but this would have been a good option for the cheap ones

wineisfine 10y ago

I tried (bought) sendy.co and it seems quite buggy. I'm still looking for something self-hosted, where any SMTP provider can be plugged into, and that does not look like its 1993. As I have some non-techies that need to work with it.

chrisanthropic 10y ago
Like I said further down, give mailtrain.org a look. It's self hosted and you can use multiple SMTP providers.
- nickjj 10y ago
  The problem with mailtrain is according to them it's not viable to use in production. There's also no tests and very little documentation. I can get around that but the main thing preventing me from using it is there's no success stories associated to the application. Sendy has dozens of unbiased blog posts, and even some posts where people are sending millions of e-mails without issues. That instills a lot of confidence in using it. You would have to be a madman to trust an untrusted application with managing your email subscribers.
  - andris9 10y ago
    Just for the record, Mailtrain is used in production by several marketers with very large lists (from hundred thousand subscribers to a million subscribers) and Mailtrain seems to work fine for them
    - nickjj 10y ago
      Is there anything to read that's public where those marketers comment about using it? Have any of them done any audits on the code base and traced all incoming requests through Mailtrain vs their SES backend to ensure everything was delivered as expected?
slig 10y ago
Try Mailwizz. It has tons of features and the author is very active on the support forum.

jonathanbull 10y ago

This is really useful - DKIM/SPF/DMARC setup is way trickier than it should be. I'm a co-founder of EmailOctopus so happy to answer any questions on the integration side of things.