5 ms·

This is great but it's a bit misleading. It only store email on S3, that mean you cannot read email with your email client. Seem thing with outgoing, you uplo

This is great but it's a bit misleading. It only store email on S3, that mean you cannot read email with your email client. Seem thing with outgoing, you upload email to s3 in a certain format to be send out, not smtp protocol This is more like an API for sending and receiving. You cannot say generate SMTP credential and drop in your mail client. Also, one of the limitation with AWS SES is you cannot sent FROM or RETURN-PATH to anything, even in production mode, you can only set frm or return-path to verified domain. I have an internal AWS SES/Lambda stacks that I used for email forwarding(only receiving) which use SES to accept email, trigger a lambda and use SES to send a copy to a personal inbox. Plan to cleanup it a bit and release

hnlmorg 5y ago
In fairness, you could write support for those in lambda too if you wanted. As a proof of concept though, this is interesting.
OJFord 5y ago
I do something similar to OP [0] - and currently forward to another provider (single address, but that's only the 'envelope' To; so it doesn't matter, I preserve the original whatever.to.address). That's mostly an artefact of transitioning from that provider to my own on AWS, but I stopped working on it for some time and never continued. My plan was/still is to write a client that can read 'Just a Bunch of .Eml' format, and then I can point that at S3 (or via S3FS or whatever). Everything that exists for mailbox, maildir, and what-have-you assumes you need it to act as an MTA (I think that's right) and do the actual receiving too; and is difficult/impossible to stop that and say 'no, here's a directory full of email, just be a reader'. Longer-term plan/idle thought is to have a JMAP server on Lambda. (IMAP isn't possible, because it has to look like/be HTTP to reach the Lambda.) [0] - https://github.com/OJFord/amail https://github.com/OJFord/amail
- kureikain 5y ago
  Yes, I used something like that as well. But, are you sure you can set "From" header to anything(even for email/domain that aren't verified with SES). So that on receiver side(gmail for example) it will show the original sender. In my account(in production mode, not being sandbox), I couldn't. When I attempted to set "From" header(or "Return-Path") I got these error 554 Message rejected: Email address is not verified. The following identities failed the check in region US-WEST-2: can-we-overwrite-this@my-domain.com, Test random <raindom@gmail.com>... On their example doc, using SMTP credential, they also mention this: https://docs.aws.amazon.com/ses/latest/DeveloperGuide/examples-send-using-smtp.html https://docs.aws.amazon.com/ses/latest/DeveloperGuide/exampl...
  - OJFord 5y ago
    No, you can't set it to anything, you'll fail DKIM/DMARC/SPF. Just leave it as it is! You just wrap the original email in an 'envelope' with envelope-to (you @ other provider) and envelope-from (whatever @ verified with SES). You can see this here, if it helps at all, though it's a bit DSL-y: https://github.com/OJFord/amail/blob/c602f4d36c1bb9df4da1744c6b2b4f78718613d0/lambda-smtp-relay/handler.rs#L92-L96 https://github.com/OJFord/amail/blob/c602f4d36c1bb9df4da1744...
    - kureikain 5y ago
      By envelope-from (whatever @ verified with SES), do you mean the `MAIL FROM` command in SMTP protocol or do you mean the `header FROM` in the email header? I don't know how `send_raw` work under the hood in Rust, but without a way to set `from` header. how do recipient(say an @gmail.com) address show the original from too? Did I miss something here? It would be great if somehow the email is forward as it's in so it appear in my inbox and all the information(header) is retain. > just wrap the original email in an 'envelope' It would be great if you can help clear my mind on how to do that. Say I got an email as this: FROM: original-from DKIM: dkim sign Return-Path: etc Other header: Body of email appear here How can I use AWS SES to forward that email as-is to my @gmail.com account for example while still keep in the original header intact(so DKIM/DMARC still work. SPF will be break but it's fine for me)
      - OJFord 5y ago
        Yes, I had to check that, but MAIL FROM & RCPT TO are the 'envelope' addresses. > It would be great if somehow the email is forward as it's in so it appear in my inbox and all the information(header) is retain. Yup that's exactly it. In the client reader it looks exactly the same as if it'd been sent there directly, rather than to SES and relayed on. (If you dig into the headers it's all there of course, but just from a 'consumer' perspective it all looks normal viewing the email.)
        kureikain 5y ago
StreamBright 5y ago
I would argue that an email client that read email from s3 instead instead of a mailbox is preferable.
jpalomaki 5y ago
Back in the days I remember storing emails in similar way and reading using an email client. It was called ”maildir” [1]. Emails were stored as files in folder structure. I think this goes quite nicely with the unix ideas as you could use many standard tools to work with the messages. [1] https://en.m.wikipedia.org/wiki/Maildir https://en.m.wikipedia.org/wiki/Maildir and
femto113 5y ago
You could probably cobble together a POP server on top of S3 pretty easily (e.g. using something like https://github.com/femto113/node-pop3-emitter https://github.com/femto113/node-pop3-emitter which I threw together a couple years ago for a related purpose). I'm not sure if you can get it to work in Lambda though because the POP3 protocol needs sockets, but you could definitely get it to work in a simple container in Fargate (so still "serverless").

hnlmorg 5y ago

In fairness, you could write support for those in lambda too if you wanted. As a proof of concept though, this is interesting.

OJFord 5y ago

I do something similar to OP [0] - and currently forward to another provider (single address, but that's only the 'envelope' To; so it doesn't matter, I preserve the original whatever.to.address). That's mostly an artefact of transitioning from that provider to my own on AWS, but I stopped working on it for some time and never continued. My plan was/still is to write a client that can read 'Just a Bunch of .Eml' format, and then I can point that at S3 (or via S3FS or whatever). Everything that exists for mailbox, maildir, and what-have-you assumes you need it to act as an MTA (I think that's right) and do the actual receiving too; and is difficult/impossible to stop that and say 'no, here's a directory full of email, just be a reader'. Longer-term plan/idle thought is to have a JMAP server on Lambda. (IMAP isn't possible, because it has to look like/be HTTP to reach the Lambda.) [0] - https://github.com/OJFord/amail https://github.com/OJFord/amail

kureikain 5y ago
Yes, I used something like that as well. But, are you sure you can set "From" header to anything(even for email/domain that aren't verified with SES). So that on receiver side(gmail for example) it will show the original sender. In my account(in production mode, not being sandbox), I couldn't. When I attempted to set "From" header(or "Return-Path") I got these error 554 Message rejected: Email address is not verified. The following identities failed the check in region US-WEST-2: can-we-overwrite-this@my-domain.com, Test random <raindom@gmail.com>... On their example doc, using SMTP credential, they also mention this: https://docs.aws.amazon.com/ses/latest/DeveloperGuide/examples-send-using-smtp.html https://docs.aws.amazon.com/ses/latest/DeveloperGuide/exampl...
- OJFord 5y ago
  No, you can't set it to anything, you'll fail DKIM/DMARC/SPF. Just leave it as it is! You just wrap the original email in an 'envelope' with envelope-to (you @ other provider) and envelope-from (whatever @ verified with SES). You can see this here, if it helps at all, though it's a bit DSL-y: https://github.com/OJFord/amail/blob/c602f4d36c1bb9df4da1744c6b2b4f78718613d0/lambda-smtp-relay/handler.rs#L92-L96 https://github.com/OJFord/amail/blob/c602f4d36c1bb9df4da1744...
  - kureikain 5y ago
    By envelope-from (whatever @ verified with SES), do you mean the `MAIL FROM` command in SMTP protocol or do you mean the `header FROM` in the email header? I don't know how `send_raw` work under the hood in Rust, but without a way to set `from` header. how do recipient(say an @gmail.com) address show the original from too? Did I miss something here? It would be great if somehow the email is forward as it's in so it appear in my inbox and all the information(header) is retain. > just wrap the original email in an 'envelope' It would be great if you can help clear my mind on how to do that. Say I got an email as this: FROM: original-from DKIM: dkim sign Return-Path: etc Other header: Body of email appear here How can I use AWS SES to forward that email as-is to my @gmail.com account for example while still keep in the original header intact(so DKIM/DMARC still work. SPF will be break but it's fine for me)
    - OJFord 5y ago
      Yes, I had to check that, but MAIL FROM & RCPT TO are the 'envelope' addresses. > It would be great if somehow the email is forward as it's in so it appear in my inbox and all the information(header) is retain. Yup that's exactly it. In the client reader it looks exactly the same as if it'd been sent there directly, rather than to SES and relayed on. (If you dig into the headers it's all there of course, but just from a 'consumer' perspective it all looks normal viewing the email.)
      - kureikain 5y ago

StreamBright 5y ago

I would argue that an email client that read email from s3 instead instead of a mailbox is preferable.

jpalomaki 5y ago

Back in the days I remember storing emails in similar way and reading using an email client. It was called ”maildir” [1]. Emails were stored as files in folder structure. I think this goes quite nicely with the unix ideas as you could use many standard tools to work with the messages. [1] https://en.m.wikipedia.org/wiki/Maildir https://en.m.wikipedia.org/wiki/Maildir and

femto113 5y ago

You could probably cobble together a POP server on top of S3 pretty easily (e.g. using something like https://github.com/femto113/node-pop3-emitter https://github.com/femto113/node-pop3-emitter which I threw together a couple years ago for a related purpose). I'm not sure if you can get it to work in Lambda though because the POP3 protocol needs sockets, but you could definitely get it to work in a simple container in Fargate (so still "serverless").