7 ms·

The article handwaves this by pointing out that it’s “proof-of-work”. Which is a fancy way of saying “it requires a bunch of work, continuously as the rules cha

by akerl_ 7y ago

The article handwaves this by pointing out that it’s “proof-of-work”. Which is a fancy way of saying “it requires a bunch of work, continuously as the rules change over time”. Which is why I’d consider it “hard” to run my own mail server. The complexity of running a daemon with a config file isn’t the issue, nor is it that any individual task of SPF/DKIM/etc is complex. It’s that I generally want to put 0% of my time into making sure my emails are being received, and I’m receiving other peoples’ emails.

brianpgordon 7y ago
It would be nice if these big mail services accepted actual proof of work as a way to guarantee that your mail gets through. Maybe someone could extend SMTP with a bitcoin-like challenge where the server sends the client a nonce and a difficulty factor, and the client has to reply with a suffix that you can append to the nonce so that the whole string hashes to something with the corresponding number of zeroes at the end. People sending personal emails will be happy to burn a penny of CPU time as "postage" but spammers won't be able to send their spam profitably, especially if you increase the difficulty factor based on how many emails you've sent in the last hour.
- esotericn 7y ago
  https://en.wikipedia.org/wiki/Hashcash https://en.wikipedia.org/wiki/Hashcash > Hashcash was proposed in 1997 by Adam Back[1] and described more formally in Back's paper "Hashcash - A Denial of Service Counter-Measure".[2]
  - brianpgordon 7y ago
    I guess the idea was too obvious to be original. I'm surprised that it inspired Bitcoin instead of the other way around though.
- deleted 7y ago
  [deleted]
- jcranmer 7y ago
  > People sending personal emails will be happy to burn a penny of CPU time as "postage" but spammers won't be able to send their spam profitably The problem with this idea is that spammers don't use their own computer, they borrow other people's computers to send the spam. It also heavily penalizes people who legitimately need to send bulk email.
  - Nullabillity 7y ago
    > It also heavily penalizes people who legitimately need to send bulk email. Aka spammers.
    - dasil003 7y ago
      Believe it or not there are email newsletters and mailing lists that people legitimately want to receive.
      - jcranmer 7y ago
        Such as the Linux kernel mailing list.
        amelius 7y ago
        Yeah, but for bulk-mail one could introduce "whitelists".
    - zbentley 7y ago
      (Note: deleted and moved here because I put it in reply to the wrong comment) I work for an email marketing provider (opinions my own, obviously), and I was extremely surprised how many people genuinely want to receive lots of commercial bulk marketing email. And by "want to" I don't mean "don't mark as spam", I mean "continually sign up for different marketing newsletters, and consistently use links in newsletters/marketing emails to make purchases from multiple companies". It's an alien phenomenon to me, and it may be a minority of bulk email, but it definitely is useful to a lot of people. Maybe it's their equivalent of coupon-clipping-based shopping, or maybe they just like learning about things to buy in that format? I have no idea. That aside, the last thing an email marketing platform wants to do is send spam. Every time someone marks an email such a provider sends as spam, the reputation of their sending addresses is damaged, and therefore their ability to deliver mail to aforementioned people who want to receive them (and therefore the ability of the marketing platform to make money by facilitating that kind of desired email) is jeopardized. TL;dr there's bulk marketing email and then there's spam, and surprisingly little overlap (and overlapping incentives) between the people who send each.
- fanf2 7y ago
  Proof of work proves not to work (2004) http://www.cl.cam.ac.uk/~rnc1/proofwork.pdf http://www.cl.cam.ac.uk/~rnc1/proofwork.pdf
  - brianpgordon 7y ago
    The paper's argument assumes that the PoW scheme is required though. It could be an option which just guarantees that individual mail can get through, while still keeping the usual reputation-based spam filtering for clients which don't want to participate.
    - jcranmer 7y ago
      If there's one lesson from the history of spamming, it's that spammers are among the fastest adopters of anti-spam technologies. If PoW meant that spammers could only send 1/100th the email, but the email was 1000× more likely to actually show up in an inbox, spammers would jump on it in a heartbeat.
      - brianpgordon 7y ago
        That doesn't make sense. Each email is only worth so much to a spammer. They can't lose money on every email and make up for it with volume.
        jcranmer 7y ago
        The value to spammers isn't in email delivered, it's the actual user engagement at the end of the process. A million messages delivered to the inbox is worth far more than a billion messages delivered to the spam folder.
        scarejunba 7y ago
        Yeah, but if the marginal cost of the next message is higher than the marginal expected increase in revenue then they won’t. It doesn’t matter optimizing inbox delivery rates if per-delivery you lose money.
- prirun 7y ago
  It's an interesting idea, but wouldn't this be easy to abuse though, like Gmail could force Fastmail to do huge proofs of work before accepting their mail?
jdboyd 7y ago
In my experience of over 15 years, the "rules" have only changed 3 times since I started. First to require good reverse DNS, then to require SPF, last to require DKIM. The transition on each of those was generally several years with plenty of publicity.
- bt848 7y ago
  That hardly covers it. The problem is that who "you" are, your identity as a sender, is not entirely within your control. The big mail operators figure in a ton of stuff, like the reputation of your subnet - /24 maybe, or /96, it's an approximation - or the reputation of your AS, your domain registrar, all kinds of stuff. If the guy on the next IP over is sending spam that's going to pollute your reputation. If you have your own ASN and your information is registered with reputable parties in lawful countries, you might not have much to worry about. If you are using a cheap cloud VPS service whose network is registered to a Romanian holding company, nobody will deliver your mail.
- toomim 7y ago
  You're forgetting DMARC.
  - teddyh 7y ago
    If you already have SPF and/or DKIM, adding DMARC is a matter of setting one static DNS record and never changing it again. Hardly difficult.
  - scarejunba 7y ago
    I don’t have DMARC setup and can still send to Gmail.
amelius 7y ago
> Which is a fancy way of saying “it requires a bunch of work, continuously as the rules change over time”. Wouldn't this be a good task for an open-source package to handle? If you'd just updated the package regularly, your mail would always be sent according to the rules. No need for any third-party handling your mail.
- jasode 7y ago
  >Wouldn't this be a good task for an open-source package to handle? If you'd just updated the package regularly, your mail would always be sent according to the rules. Some of the "rules" for reliable outgoing email delivery cannot be encoded inside the source code files of a github repo, or email setup bash script, or a Docker image, or a EC2 virtual image, etc. An example of an unspecified "rule" outside the boundaries of a local machine holding the email server is recovering from an unexpected blacklisting of ip addresses. Example.[1] A preconfigured Docker image of Dovecot isn't going to magically ask Amazon support staff why the ip address is blacklisted and/or move the server to a different ISP etc. If a bad actor (that you are unaware of and have no control over) happens to share your ip address block and sends out spam which then causes Gmail/Hotmail/Yahoo to reject your server's emails, there's no open-source programming code that can detect and fix those problems happening outside of your control. Or put another way, if HonestJoeBlow could download a constantly updated Docker image that has the so-called "correct rules" for sent emails to always be accepted by Gmail, it would mean the DishonestSpammers could also download that same Docker image to get their emails delivered. The issue is that "trust" in the email ecosystem is an emergent property among participants and therefore, it can't all be embedded inside email server configs, or in a DIY blog article trying to explain the exact 12 steps to make email delivery work perfectly with all receivers. Eventually, something can break (because a receiver changes their idea of "trust" and "spam") and it requires troubleshooting & debugging to get email delivery working again. A tldr would be: You can't put "sender reputation" into a downloadable software package. [1] https://forums.aws.amazon.com/message.jspa?messageID=724690 https://forums.aws.amazon.com/message.jspa?messageID=724690
  - temptemptemp111 7y ago
    However, would be sweet for a bunch of mailserver experts to converge on a docker-compose or swarm file for an optimal setup.

by akerl_ 7y ago

brianpgordon 7y ago

It would be nice if these big mail services accepted actual proof of work as a way to guarantee that your mail gets through. Maybe someone could extend SMTP with a bitcoin-like challenge where the server sends the client a nonce and a difficulty factor, and the client has to reply with a suffix that you can append to the nonce so that the whole string hashes to something with the corresponding number of zeroes at the end. People sending personal emails will be happy to burn a penny of CPU time as "postage" but spammers won't be able to send their spam profitably, especially if you increase the difficulty factor based on how many emails you've sent in the last hour.

esotericn 7y ago
https://en.wikipedia.org/wiki/Hashcash https://en.wikipedia.org/wiki/Hashcash > Hashcash was proposed in 1997 by Adam Back[1] and described more formally in Back's paper "Hashcash - A Denial of Service Counter-Measure".[2]
- brianpgordon 7y ago
  I guess the idea was too obvious to be original. I'm surprised that it inspired Bitcoin instead of the other way around though.
deleted 7y ago
[deleted]
jcranmer 7y ago
> People sending personal emails will be happy to burn a penny of CPU time as "postage" but spammers won't be able to send their spam profitably The problem with this idea is that spammers don't use their own computer, they borrow other people's computers to send the spam. It also heavily penalizes people who legitimately need to send bulk email.
- Nullabillity 7y ago
  > It also heavily penalizes people who legitimately need to send bulk email. Aka spammers.
  - dasil003 7y ago
    Believe it or not there are email newsletters and mailing lists that people legitimately want to receive.
    - jcranmer 7y ago
      Such as the Linux kernel mailing list.
      - amelius 7y ago
        Yeah, but for bulk-mail one could introduce "whitelists".
  - zbentley 7y ago
    (Note: deleted and moved here because I put it in reply to the wrong comment) I work for an email marketing provider (opinions my own, obviously), and I was extremely surprised how many people genuinely want to receive lots of commercial bulk marketing email. And by "want to" I don't mean "don't mark as spam", I mean "continually sign up for different marketing newsletters, and consistently use links in newsletters/marketing emails to make purchases from multiple companies". It's an alien phenomenon to me, and it may be a minority of bulk email, but it definitely is useful to a lot of people. Maybe it's their equivalent of coupon-clipping-based shopping, or maybe they just like learning about things to buy in that format? I have no idea. That aside, the last thing an email marketing platform wants to do is send spam. Every time someone marks an email such a provider sends as spam, the reputation of their sending addresses is damaged, and therefore their ability to deliver mail to aforementioned people who want to receive them (and therefore the ability of the marketing platform to make money by facilitating that kind of desired email) is jeopardized. TL;dr there's bulk marketing email and then there's spam, and surprisingly little overlap (and overlapping incentives) between the people who send each.
fanf2 7y ago
Proof of work proves not to work (2004) http://www.cl.cam.ac.uk/~rnc1/proofwork.pdf http://www.cl.cam.ac.uk/~rnc1/proofwork.pdf
- brianpgordon 7y ago
  The paper's argument assumes that the PoW scheme is required though. It could be an option which just guarantees that individual mail can get through, while still keeping the usual reputation-based spam filtering for clients which don't want to participate.
  - jcranmer 7y ago
    If there's one lesson from the history of spamming, it's that spammers are among the fastest adopters of anti-spam technologies. If PoW meant that spammers could only send 1/100th the email, but the email was 1000× more likely to actually show up in an inbox, spammers would jump on it in a heartbeat.
    - brianpgordon 7y ago
      That doesn't make sense. Each email is only worth so much to a spammer. They can't lose money on every email and make up for it with volume.
      - jcranmer 7y ago
        The value to spammers isn't in email delivered, it's the actual user engagement at the end of the process. A million messages delivered to the inbox is worth far more than a billion messages delivered to the spam folder.
        scarejunba 7y ago
        Yeah, but if the marginal cost of the next message is higher than the marginal expected increase in revenue then they won’t. It doesn’t matter optimizing inbox delivery rates if per-delivery you lose money.
prirun 7y ago
It's an interesting idea, but wouldn't this be easy to abuse though, like Gmail could force Fastmail to do huge proofs of work before accepting their mail?

jdboyd 7y ago

In my experience of over 15 years, the "rules" have only changed 3 times since I started. First to require good reverse DNS, then to require SPF, last to require DKIM. The transition on each of those was generally several years with plenty of publicity.

bt848 7y ago
That hardly covers it. The problem is that who "you" are, your identity as a sender, is not entirely within your control. The big mail operators figure in a ton of stuff, like the reputation of your subnet - /24 maybe, or /96, it's an approximation - or the reputation of your AS, your domain registrar, all kinds of stuff. If the guy on the next IP over is sending spam that's going to pollute your reputation. If you have your own ASN and your information is registered with reputable parties in lawful countries, you might not have much to worry about. If you are using a cheap cloud VPS service whose network is registered to a Romanian holding company, nobody will deliver your mail.
toomim 7y ago
You're forgetting DMARC.
- teddyh 7y ago
  If you already have SPF and/or DKIM, adding DMARC is a matter of setting one static DNS record and never changing it again. Hardly difficult.
- scarejunba 7y ago
  I don’t have DMARC setup and can still send to Gmail.

amelius 7y ago

> Which is a fancy way of saying “it requires a bunch of work, continuously as the rules change over time”. Wouldn't this be a good task for an open-source package to handle? If you'd just updated the package regularly, your mail would always be sent according to the rules. No need for any third-party handling your mail.

jasode 7y ago
>Wouldn't this be a good task for an open-source package to handle? If you'd just updated the package regularly, your mail would always be sent according to the rules. Some of the "rules" for reliable outgoing email delivery cannot be encoded inside the source code files of a github repo, or email setup bash script, or a Docker image, or a EC2 virtual image, etc. An example of an unspecified "rule" outside the boundaries of a local machine holding the email server is recovering from an unexpected blacklisting of ip addresses. Example.[1] A preconfigured Docker image of Dovecot isn't going to magically ask Amazon support staff why the ip address is blacklisted and/or move the server to a different ISP etc. If a bad actor (that you are unaware of and have no control over) happens to share your ip address block and sends out spam which then causes Gmail/Hotmail/Yahoo to reject your server's emails, there's no open-source programming code that can detect and fix those problems happening outside of your control. Or put another way, if HonestJoeBlow could download a constantly updated Docker image that has the so-called "correct rules" for sent emails to always be accepted by Gmail, it would mean the DishonestSpammers could also download that same Docker image to get their emails delivered. The issue is that "trust" in the email ecosystem is an emergent property among participants and therefore, it can't all be embedded inside email server configs, or in a DIY blog article trying to explain the exact 12 steps to make email delivery work perfectly with all receivers. Eventually, something can break (because a receiver changes their idea of "trust" and "spam") and it requires troubleshooting & debugging to get email delivery working again. A tldr would be: You can't put "sender reputation" into a downloadable software package. [1] https://forums.aws.amazon.com/message.jspa?messageID=724690 https://forums.aws.amazon.com/message.jspa?messageID=724690
- temptemptemp111 7y ago
  However, would be sweet for a bunch of mailserver experts to converge on a docker-compose or swarm file for an optimal setup.