7 ms·

We really need a better solution to fighting spam. https://github.com/jimktrains/email_ng https://github.com/jimktrains/email_ng is an overly complicated idea

We really need a better solution to fighting spam. https://github.com/jimktrains/email_ng https://github.com/jimktrains/email_ng is an overly complicated idea I had once, but the basic gist is that the receiver could give the sender a signed receiver email address that is for the sender's email address. This way, transactional email (and sure, I guess marketing from your company) would be able to get through and the email given out wouldn't be able to be used if sold or stolen, as the receiving mail server would reject it because of a bad signature (bad sender, and coupled with DKIM and SPF, a malicious user wouldn't be able/allowed to spoof the sender's email).

chatmasta 10y ago
This sounds like a cryptographic way to achieve the aliasing scheme many people already use, e.g "chatmasta+hackernews@gmail.com". Your proposal requires at least the status quo (aliased email address at signup), but the problem is it also requires cooperation of the receiver's email provider (e.g. gmail) and the sender's email provider. You are unlikely to ever see any new standard adopted by all major email providers, unless it came through a standards committee. But your idea is interesting. Perhaps you could lower the cost of adoption by replacing the dependency on TLS/SSL with some sort of pgp signing scheme. This way all the "protocol" happens within the message body, so email providers do not need to adopt a new standard. As long as at least one website and one user implement the protocol, it can work without any cooperation from third parties. As an aside, it would also be nice if password managers included functionality around generating temporary/isolated email addresses.
- itissid 10y ago
  Actually curious, so if I understand you correctly what you are saying is that a receiver had some software generated an address like johndoe+A16789HFF...@gmail.com and then sent it to the sender for use, that would require email providers cooperation. I am not clear on the "cooperation" parts work. I THINK its basically that email providers block emails based on other criteria like DNSBL. And the "cooperation" bit is to get the providers to manage these cryptographic email addresses like guard against their possible misuse?
  - jimktrains2 10y ago
    That's more-or-less what I was thinking. The receiver would require understanding of the signature to white-list the message; the sender just sends to an email address as they would normally. No or invalid sig would fall back to our current anti-spam methods.
- StavrosK 10y ago
  There are services like 33mail.com, which I've used for years, which do exactly that. You give a service an email like servicename@chatmasta.33mail.com (or a custom domain, if you're on a paid plan), and then you can block/unblock/track emails per username.
- masukomi 10y ago
  why not just add +sitename to your email when you sign up..... eg me@example.com becomes me+flickr@example.com ? it's just a comment that will be ignored. I don't feel that we need software support to help with that. Your password manager will store that address (since it's probably your username) and help you fill it in / remember it when you come back to the site.
  - hk__2 10y ago
    > it's just a comment that will be ignored. It’s not a comment. Some mail services allow you to have aliases in the form of normal-username+something@service.com but that doesn’t mean all do. If it were a comment the address +@example.com would be invalid but it’s not. Comments in email addresses are written in parentheses like username(i'm_a_comment)@example.com.
AJ007 10y ago
I think overall ISPs do a very good job at fighting spam. Compared to a decade and a half ago, I would call it a minimal problem. My greatest fear is that Google and Facebook will co-opt email and charge money for companies to get access. Google has already made some progress with this by selling ads in Gmail that look like email while pushing commercial messages in to other inboxes. This is bad because it penalizes companies with low margins while shifting the advantage to those who are able to squeeze more money out of the users (you can see the effect of this on Google Search, Facebook, and now Amazon. It isn't pretty.)
- drdaeman 10y ago
  Correction: large^W giant mail systems, that can observe a significant portion of world's mail, and have a lot of R&D resources, do a very good job at fighting spam (in their own systems). A DIY home mailserver will still let, like, 5-15% of spam emails slip through.
  - jimktrains2 10y ago
    Even with Gmail I still find enough false-positives on spam to make me troll through it daily to make sure important emails don't get sent there.
    - user5994461 10y ago
      Gmail does an incredible job. The last few times I went to spam and saw emails that shouldn't be in there, a careful inspection revealed they were actually very well crafted phishing messages. Ever since, I don't bother checking the spam folder. Too dangerous. Gmail is smarter than me.
      - jimktrains2 10y ago
        No, legit emails end up there every day for me.
        10y ago
criddell 10y ago
> We really need a better solution to fighting spam. I rarely receive email spam these days. The spam I get is on my voice line. I'm getting several robocalls per day. If you have the skills to work on this stuff, I'd love to see a solution to phone call spam because email spam feels solved.
- apatters 10y ago
  My situation is the exact opposite. I almost never get robocalls but email spam is getting worse every month!
- jimktrains2 10y ago
  I get more spam and more robocalls every single day. I can't rely on even Gmail's spam filter, because it has a non-negligible amount of false-positives.
- masukomi 10y ago
  yes, but as someone who's worked on the sending of emails side of this i can assure you that the techniques for fighting spam are TOO effective. There are many legit emails that go into black holes before any standard idea of spam filtering comes into play. I wrote an embeddable smtp server so that you could have an app that sent email without ever asking your users to put in their SMTP server info... it was USELESS because of SPF filtering. The blacklisting of whole IP ranges means that no app could use it on a home computer because all those IPs are banned. we definitely need a better solution to fighting spam.
  - Neliquat 10y ago
    I hear this all the time, but the culprit is always someone sending spam, who does not consider it spam. Usually stuff like missing unsubscribes, spammy language, sending bulk to bought email lists, etc.
    - jimktrains2 10y ago
      For a company I worked for, it was transactional emails (recipes) that would get blackholed. So while you're probably right that a lot of people doing legit marketing, but not 100% correctly often complain, it's also people doing legit work as well.
derefr 10y ago
I pondered something similar once. I'd start with the "email postage" proposal—the one where either a proof-of-work or physical payment (which can also be digital, e.g. a Bitcoin private key) is required to get an MTA to take your message. This proposal was declared unworkable due to rendering high-volume transactional email impossible. But then: 1. as you say here, divide everyone's email account into a set of "channels" (i.e. subaddresses in a+b@c form, that each have a corresponding message signing key), with one or more published or well-known channels, and then individual private channels for each contact/list, or for each conversation(!); 2. make MTAs aware of "channels", and extend both SMTP and email web services to allow users to configure their provider's end-of-line MTA, over-the-wire, to set the amount of "postage" required to message each channel they own. (This way, the MTA doesn't have to be aware of the distinction between private and public channels; they're just destination addresses with a stored config parameter.) 3. make the MTAs that use channels, reject any Internet message directed to the base address without a channel. (Messages generated by the MTA itself can arrive without a channel.) 4. Set the clients' default new-channel configurations such that public channels have a cost, and private channels are free. 5. In email clients, add a "Subscribe" or "Allow" action-bar item to messages that identify themselves as email-confirmation/newsletter-opt-in emails (presumably with a header), that, when clicked, creates a new channel for the sender, and replies to the message with the signing key you want them to use attached. (All this would be hidden from view; the reply message wouldn't end up in your Sent Items.) 6. In transactional-email-sending services like Sendgrid/Postmark/etc., create a distinction between "opt-in messages" and "ongoing transactional messages"; allow each account to send one "opt-in" email to each previously-unknown destination-address (and charge for this); but then, for that account, put that destination-address into holding state, where the account can't send them any "ongoing transactional messages", until the email-sending service receives the user's conversation key. (You probably want to allow accounts to re-send the opt-in email after a 24-hour-cooldown, though. Though they'd have to pay again!) 7. In any product/service backend that uses a transactional-email service, consider new signups unconfirmed until the transactional-service reports (by polling or webhook or whatever) that the user has replied with a key and is now in the "authenticated and free to send to" state. A bit complicated in the initial changes, but in the ongoing state it's nearly ideal: it costs money to initiate contact with an address, or to continue pestering an address that doesn't want to speak with you, but not to send messages to someone that wants to hear from you. (Though, a user can "unsubscribe" from your list simply by telling their MTA to begin charging you to deliver to their channel again—maybe, UI-wise, by just deleting the channel. Transactional-email sending services could detect the bounced-with-payment-required delivery error, and put the user automatically into an "unsubscribed" state, which could webhook you to let you know!)

chatmasta 10y ago

This sounds like a cryptographic way to achieve the aliasing scheme many people already use, e.g "chatmasta+hackernews@gmail.com". Your proposal requires at least the status quo (aliased email address at signup), but the problem is it also requires cooperation of the receiver's email provider (e.g. gmail) and the sender's email provider. You are unlikely to ever see any new standard adopted by all major email providers, unless it came through a standards committee. But your idea is interesting. Perhaps you could lower the cost of adoption by replacing the dependency on TLS/SSL with some sort of pgp signing scheme. This way all the "protocol" happens within the message body, so email providers do not need to adopt a new standard. As long as at least one website and one user implement the protocol, it can work without any cooperation from third parties. As an aside, it would also be nice if password managers included functionality around generating temporary/isolated email addresses.

itissid 10y ago
Actually curious, so if I understand you correctly what you are saying is that a receiver had some software generated an address like johndoe+A16789HFF...@gmail.com and then sent it to the sender for use, that would require email providers cooperation. I am not clear on the "cooperation" parts work. I THINK its basically that email providers block emails based on other criteria like DNSBL. And the "cooperation" bit is to get the providers to manage these cryptographic email addresses like guard against their possible misuse?
- jimktrains2 10y ago
  That's more-or-less what I was thinking. The receiver would require understanding of the signature to white-list the message; the sender just sends to an email address as they would normally. No or invalid sig would fall back to our current anti-spam methods.
StavrosK 10y ago
There are services like 33mail.com, which I've used for years, which do exactly that. You give a service an email like servicename@chatmasta.33mail.com (or a custom domain, if you're on a paid plan), and then you can block/unblock/track emails per username.
masukomi 10y ago
why not just add +sitename to your email when you sign up..... eg me@example.com becomes me+flickr@example.com ? it's just a comment that will be ignored. I don't feel that we need software support to help with that. Your password manager will store that address (since it's probably your username) and help you fill it in / remember it when you come back to the site.
- hk__2 10y ago
  > it's just a comment that will be ignored. It’s not a comment. Some mail services allow you to have aliases in the form of normal-username+something@service.com but that doesn’t mean all do. If it were a comment the address +@example.com would be invalid but it’s not. Comments in email addresses are written in parentheses like username(i'm_a_comment)@example.com.

AJ007 10y ago

I think overall ISPs do a very good job at fighting spam. Compared to a decade and a half ago, I would call it a minimal problem. My greatest fear is that Google and Facebook will co-opt email and charge money for companies to get access. Google has already made some progress with this by selling ads in Gmail that look like email while pushing commercial messages in to other inboxes. This is bad because it penalizes companies with low margins while shifting the advantage to those who are able to squeeze more money out of the users (you can see the effect of this on Google Search, Facebook, and now Amazon. It isn't pretty.)

drdaeman 10y ago
Correction: large^W giant mail systems, that can observe a significant portion of world's mail, and have a lot of R&D resources, do a very good job at fighting spam (in their own systems). A DIY home mailserver will still let, like, 5-15% of spam emails slip through.
- jimktrains2 10y ago
  Even with Gmail I still find enough false-positives on spam to make me troll through it daily to make sure important emails don't get sent there.
  - user5994461 10y ago
    Gmail does an incredible job. The last few times I went to spam and saw emails that shouldn't be in there, a careful inspection revealed they were actually very well crafted phishing messages. Ever since, I don't bother checking the spam folder. Too dangerous. Gmail is smarter than me.
    - jimktrains2 10y ago
      No, legit emails end up there every day for me.
      - 10y ago

criddell 10y ago

> We really need a better solution to fighting spam. I rarely receive email spam these days. The spam I get is on my voice line. I'm getting several robocalls per day. If you have the skills to work on this stuff, I'd love to see a solution to phone call spam because email spam feels solved.

apatters 10y ago
My situation is the exact opposite. I almost never get robocalls but email spam is getting worse every month!
jimktrains2 10y ago
I get more spam and more robocalls every single day. I can't rely on even Gmail's spam filter, because it has a non-negligible amount of false-positives.
masukomi 10y ago
yes, but as someone who's worked on the sending of emails side of this i can assure you that the techniques for fighting spam are TOO effective. There are many legit emails that go into black holes before any standard idea of spam filtering comes into play. I wrote an embeddable smtp server so that you could have an app that sent email without ever asking your users to put in their SMTP server info... it was USELESS because of SPF filtering. The blacklisting of whole IP ranges means that no app could use it on a home computer because all those IPs are banned. we definitely need a better solution to fighting spam.
- Neliquat 10y ago
  I hear this all the time, but the culprit is always someone sending spam, who does not consider it spam. Usually stuff like missing unsubscribes, spammy language, sending bulk to bought email lists, etc.
  - jimktrains2 10y ago
    For a company I worked for, it was transactional emails (recipes) that would get blackholed. So while you're probably right that a lot of people doing legit marketing, but not 100% correctly often complain, it's also people doing legit work as well.

derefr 10y ago

I pondered something similar once. I'd start with the "email postage" proposal—the one where either a proof-of-work or physical payment (which can also be digital, e.g. a Bitcoin private key) is required to get an MTA to take your message. This proposal was declared unworkable due to rendering high-volume transactional email impossible. But then: 1. as you say here, divide everyone's email account into a set of "channels" (i.e. subaddresses in a+b@c form, that each have a corresponding message signing key), with one or more published or well-known channels, and then individual private channels for each contact/list, or for each conversation(!); 2. make MTAs aware of "channels", and extend both SMTP and email web services to allow users to configure their provider's end-of-line MTA, over-the-wire, to set the amount of "postage" required to message each channel they own. (This way, the MTA doesn't have to be aware of the distinction between private and public channels; they're just destination addresses with a stored config parameter.) 3. make the MTAs that use channels, reject any Internet message directed to the base address without a channel. (Messages generated by the MTA itself can arrive without a channel.) 4. Set the clients' default new-channel configurations such that public channels have a cost, and private channels are free. 5. In email clients, add a "Subscribe" or "Allow" action-bar item to messages that identify themselves as email-confirmation/newsletter-opt-in emails (presumably with a header), that, when clicked, creates a new channel for the sender, and replies to the message with the signing key you want them to use attached. (All this would be hidden from view; the reply message wouldn't end up in your Sent Items.) 6. In transactional-email-sending services like Sendgrid/Postmark/etc., create a distinction between "opt-in messages" and "ongoing transactional messages"; allow each account to send one "opt-in" email to each previously-unknown destination-address (and charge for this); but then, for that account, put that destination-address into holding state, where the account can't send them any "ongoing transactional messages", until the email-sending service receives the user's conversation key. (You probably want to allow accounts to re-send the opt-in email after a 24-hour-cooldown, though. Though they'd have to pay again!) 7. In any product/service backend that uses a transactional-email service, consider new signups unconfirmed until the transactional-service reports (by polling or webhook or whatever) that the user has replied with a key and is now in the "authenticated and free to send to" state. A bit complicated in the initial changes, but in the ongoing state it's nearly ideal: it costs money to initiate contact with an address, or to continue pestering an address that doesn't want to speak with you, but not to send messages to someone that wants to hear from you. (Though, a user can "unsubscribe" from your list simply by telling their MTA to begin charging you to deliver to their channel again—maybe, UI-wise, by just deleting the channel. Transactional-email sending services could detect the bounced-with-payment-required delivery error, and put the user automatically into an "unsubscribed" state, which could webhook you to let you know!)