5 ms·

> despite sending an email being an utterly trivial, long solved problem. It is absolutely not. Even if you are not considering the whole spam thing. Sending

by Eikon 7y ago

> despite sending an email being an utterly trivial, long solved problem. It is absolutely not. Even if you are not considering the whole spam thing. Sending an email with the multitude of clients, MIME which is very complex in itself, network issues, message queuing, retries, unsubscription, bounces, providers feedback loops, rate limiting and what not is very hard. If you add the whole spam folder + blacklists situation into consideration, sending emails becomes a more than painful thing you don't want to deal with.

kick 7y ago
Sending an email is absolutely a solved problem. I can send a message to my address in 2020 using a 2003 Fedora Core 1 box without a domain using only what's part of the POSIX standard, and I'll receive it just fine. I just did a few days ago, actually. It even got past my provider's spam filter. I could probably have done it using earlier software than 2003-era mail/mailx, but that was the quickest way I could find to send an attachment given the software on hand.
- ebiester 7y ago
  You can still do that. However, I, as a receiver of email, get thousands of spam messages a day. So, I use services that block them before they even reach my view, or get sent directly to the spam folder. If you want to send me an email, that's solved. If you want to send it to ten thousand people on your mailing list, and not have it end up in the spam folder, you may have more work.
  - bigtunacan 7y ago
    Please explain the difference to me. Mailgun is sending tons and tons of spam email; what is so special about them that they aren't ending up in people's spam/junk folders?
    - slig 7y ago
      They probably send much, much more valid emails, respond quickly to abuse reports, and since they're large/known enough, they won't get blacklisted because that would be disruptive to large, legit business.
    - aseipp 7y ago
      Mail providers like Mailgun live or die based on their reputation. If they as a sender get marked as spam by "The Algorithms", then their product is useless. They almost certainly respond to abuse reports quickly and have enough reputation, and enough legitimate clients, that they have "leeway". But that leeway isn't unlimited. There was a big fit thrown on this website a year or two ago when one transactional mail provider (MailChimp?) said they were banning crypto companies with ICOs from their service. They literally put "ICOs are not welcome" in their Terms of Service. Why? Because first off, ICOs are scams. And that means they buy email lists, and then spam the shit out of people with "buy into our ICO!" emails, which get immediately shitcanned into the spam folder by 99.99% of users -- they're no different than Nigerian Princes, as far as most people care. That behavior tanks the reputation of the sender, and you cannot reason with The Algorithm once it "recognizes" you as spam. That kind of problem is not a minor inconvenience to companies like Mailgun, it's an existential threat.
      - lolc 7y ago
        They really care with their reputation on the line. Here for example is their response to the only complaint I've had to send Mailgun: "Thanks for sending this over. I was able to tie this sender to a compromised account, which has been disabled in our system to prevent further messages." Pity we'll have to move off them. It was good while it lasted.
  - staticautomatic 7y ago
    This raises what's really the ultimate question, which is whether and how much individuals should be made to suffer so that advertisers can send spam. Certainly the recipients are not responsible for the advertisers' decisions to spam, or more charitably, to blur the lines. Yet it's individuals who pay, in this case literally, for the advertisers' privilege of spamming them. It's extremely fucked.
    - edmundsauto 7y ago
      I think this is what happens in open systems. The alternative is walled gardens, like FB messages. Even those suffer from spam, although it's much less of the wild west.
      - mc3 7y ago
        The alternative is simple: For stranger to stranger communication, both business and non-business set up a web form. With a challenge if needed. For friend to friend communication, use email, friends' address in on a white list. For business to consumer communication, white list is again used. For spammer to anyone communication, including business lists that refuse to take you off, not on white list so doesn't get in. Bounce response with reason "non on white list" It's a cultural change though: an email address won't get through unless you are unblocked, but technically easy.
        sokoloff 7y ago
        What's the easy/low-friction way for Joe Random Consumer to manage their whitelists? An email comes in from a sender address which isn't on the whitelist, but is (in fact) a friend. What's the handling? An email comes in from a sender address which isn't on the whitelist, but is not (in fact) a friend. What's the handling? Do I never hear from the first friend? Or do I get bombarded with "hey, read this email and see if the sender is a friend" a bunch of times?
        mc3 7y ago
        Fair game. You got me, I have no nice answer other than phone your friend. I'll stop digging. I'm not going to do what this guy did: https://news.ycombinator.com/item?id=16862739 https://news.ycombinator.com/item?id=16862739
  - mirimir 7y ago
    That cuts both ways. You might miss something valuable. Perhaps from a long-lost relative. Or a prospective client or employer. And once you're actually looking through the spam folder, what good is it really?
- space_ghost 7y ago
  You're both right, but you're solving different problems. Just because your message got past your spam filter doesn't mean it would get past all spam filters.
  - kick 7y ago
    It is absolutely not. Even if you are not considering the whole spam thing. They specified that it wasn't even if you could ignore the spam issue. I said that it was if you ignored the spam issue. The first post in this thread was "We've made sending electronic mail way too complicated under the guise of fighting spam." I'm aware that it won't get past every spam filter, but it gets past both Gmail's and my own provider (a smaller, non-American one), which is good enough for my own use. My claim was that sending electronic mail is a solved problem, because it is.
    - bigiain 7y ago
      > My claim was that sending electronic mail is a solved problem, because it is. Agreed. I mean, I can _probably_ remember enough to be able to send mail using just telnet without even needing to look anything up. (And that mail, without any mime parts or urls - it very very likely to not fall into any of the spam filtering on my inbound email accounts...)
      - greglindahl 7y ago
        In the modern era, lots of mailservers run software that looks for weirdnesses in incoming email. I don't know, but I'd suspect that the typing delays of using telnet to send mail are likely to trigger their defenses.
        bigiain 7y ago
        SO I just tested. After working out that on macOS I now need to use nc -c instead of telnet... I _can_ send mail manually, but it _does_ end up in gmails's spam folder saying "Why is this message in spam? It is similar to messages that were identified as spam in the past." Iain-2:~ iain$ nc -c gmail-smtp-in.l.google.com 25 220 mx.google.com ESMTP k5si5682618pls.209 - gsmtp EHLO elided.org 250-mx.google.com at your service, [202.171.181.100] 250-SIZE 157286400 250-8BITMIME 250-STARTTLS 250-ENHANCEDSTATUSCODES 250-PIPELINING 250-CHUNKING 250 SMTPUTF8 MAIL FROM: <bigiain@elided.org> 250 2.1.0 OK k5si5682618pls.209 - gsmtp RCPT TO: <bigiain@gmail.com> 250 2.1.5 OK k5si5682618pls.209 - gsmtp DATA 354 Go ahead k5si5682618pls.209 - gsmtp Subject: Test From: bigiain@elided.org To: bigiain@gmail.com Date: Fri, 31 Jan 2020 13:07:00 +1100 Foo baz nah . 250 2.0.0 OK 1580436443 k5si5682618pls.209 - gsmtp QUIT 221 2.0.0 closing connection k5si5682618pls.209 - gsmtp Iain-2:~ iain$
- cosmotic 7y ago
  Most ISPs block SMTP traffic, which creates a pretty significant barrier to sending email. Even VPS services send through a secret gateways with all sorts of filters and controls on it.
- gwittel 7y ago
  Sending at scale is the problem. A provider like Mailgun (aka an ESP), is a huge target for abuse. Free and low priced tiers are tricky since they need to draw in customers, but then get heavily abused. tl;dr: Abuse ruins it for everyone. I work on the filtering side of the world so I see a lot of the challenges that even purportedly good actors face. Each provider does better or worse at controlling abuse. These things all have a cost: * IPs -- You need to send from a variety of IPs. They have warmup time before they can be safely used. One bad guy can burn that IP and everyone who shares. Most ESPs aren't going to have dedicated IPs for each (or those are for high tiers). * Dmarc/dkim/spf -- Passing validation can be hard since it requires your client to work with you. * Abuse reports -- Likely huge volume * Bounces -- They can count against the customer. How much is OK (e.g. AWS SES will cut you off if your bounce rate gets too high)? * Throwaway account abuse -- Huge issue, this threshold is a simple hammer (so now bad guys need ~10x more throwaway accounts). * Account takeover -- That good guy is now sending out crap. Now what? * Post-abuse cleanup -- Good luck going around and working with major providers to get yourself unblocked. Its a huge time sink. By using Mailgun, etc all of the above becomes their problem. On top of it they'll offer analytics, help crafting content that works in varying mail clients, etc. A small business type sender doesn't need much of this (other than maybe the technical help). But at some point, the scales tip.
  - webbie917 7y ago
    ^ Couldn't agree more, especially "Abuse ruins it for everyone."
  - quicksilver03 7y ago
    > IPs -- You need to send from a variety of IPs. They have warmup time before they can be safely used. One bad guy can burn that IP and everyone who shares. Most ESPs aren't going to have dedicated IPs for each (or those are for high tiers). I'm curious, is that specific to IPv4 or you see the same phenomenon with IPv6 where providers allocate /64 or /48 to customers? Or there simply isn't enough sending volume from IPv6 at the moment to warrant specific filtering?
    - gwittel 7y ago
NightlyDev 7y ago
I've sent a lot of mail in the last 14 years, and it's not hard. I spammed with invite mails to users friends back in 2008 and mail started to end up in spam for a while, but after switching to double opt-in for all mail and making unsubscribing easily available it went back to good. Add some periodic list cleaning of long time inactive subscribers and it works perfectly! It does get harder if you have to account for bad users, but if you're the only user then you can make sure everything is good. So, is it hard? That depends entirely on how wanted or unwanted the sent mail is.

by Eikon 7y ago

kick 7y ago

Sending an email is absolutely a solved problem. I can send a message to my address in 2020 using a 2003 Fedora Core 1 box without a domain using only what's part of the POSIX standard, and I'll receive it just fine. I just did a few days ago, actually. It even got past my provider's spam filter. I could probably have done it using earlier software than 2003-era mail/mailx, but that was the quickest way I could find to send an attachment given the software on hand.

ebiester 7y ago
You can still do that. However, I, as a receiver of email, get thousands of spam messages a day. So, I use services that block them before they even reach my view, or get sent directly to the spam folder. If you want to send me an email, that's solved. If you want to send it to ten thousand people on your mailing list, and not have it end up in the spam folder, you may have more work.
- bigtunacan 7y ago
  Please explain the difference to me. Mailgun is sending tons and tons of spam email; what is so special about them that they aren't ending up in people's spam/junk folders?
  - slig 7y ago
    They probably send much, much more valid emails, respond quickly to abuse reports, and since they're large/known enough, they won't get blacklisted because that would be disruptive to large, legit business.
  - aseipp 7y ago
    Mail providers like Mailgun live or die based on their reputation. If they as a sender get marked as spam by "The Algorithms", then their product is useless. They almost certainly respond to abuse reports quickly and have enough reputation, and enough legitimate clients, that they have "leeway". But that leeway isn't unlimited. There was a big fit thrown on this website a year or two ago when one transactional mail provider (MailChimp?) said they were banning crypto companies with ICOs from their service. They literally put "ICOs are not welcome" in their Terms of Service. Why? Because first off, ICOs are scams. And that means they buy email lists, and then spam the shit out of people with "buy into our ICO!" emails, which get immediately shitcanned into the spam folder by 99.99% of users -- they're no different than Nigerian Princes, as far as most people care. That behavior tanks the reputation of the sender, and you cannot reason with The Algorithm once it "recognizes" you as spam. That kind of problem is not a minor inconvenience to companies like Mailgun, it's an existential threat.
    - lolc 7y ago
      They really care with their reputation on the line. Here for example is their response to the only complaint I've had to send Mailgun: "Thanks for sending this over. I was able to tie this sender to a compromised account, which has been disabled in our system to prevent further messages." Pity we'll have to move off them. It was good while it lasted.
- staticautomatic 7y ago
  This raises what's really the ultimate question, which is whether and how much individuals should be made to suffer so that advertisers can send spam. Certainly the recipients are not responsible for the advertisers' decisions to spam, or more charitably, to blur the lines. Yet it's individuals who pay, in this case literally, for the advertisers' privilege of spamming them. It's extremely fucked.
  - edmundsauto 7y ago
    I think this is what happens in open systems. The alternative is walled gardens, like FB messages. Even those suffer from spam, although it's much less of the wild west.
    - mc3 7y ago
      The alternative is simple: For stranger to stranger communication, both business and non-business set up a web form. With a challenge if needed. For friend to friend communication, use email, friends' address in on a white list. For business to consumer communication, white list is again used. For spammer to anyone communication, including business lists that refuse to take you off, not on white list so doesn't get in. Bounce response with reason "non on white list" It's a cultural change though: an email address won't get through unless you are unblocked, but technically easy.
      - sokoloff 7y ago
        What's the easy/low-friction way for Joe Random Consumer to manage their whitelists? An email comes in from a sender address which isn't on the whitelist, but is (in fact) a friend. What's the handling? An email comes in from a sender address which isn't on the whitelist, but is not (in fact) a friend. What's the handling? Do I never hear from the first friend? Or do I get bombarded with "hey, read this email and see if the sender is a friend" a bunch of times?
        mc3 7y ago
        Fair game. You got me, I have no nice answer other than phone your friend. I'll stop digging. I'm not going to do what this guy did: https://news.ycombinator.com/item?id=16862739 https://news.ycombinator.com/item?id=16862739
- mirimir 7y ago
  That cuts both ways. You might miss something valuable. Perhaps from a long-lost relative. Or a prospective client or employer. And once you're actually looking through the spam folder, what good is it really?
space_ghost 7y ago
You're both right, but you're solving different problems. Just because your message got past your spam filter doesn't mean it would get past all spam filters.
- kick 7y ago
  It is absolutely not. Even if you are not considering the whole spam thing. They specified that it wasn't even if you could ignore the spam issue. I said that it was if you ignored the spam issue. The first post in this thread was "We've made sending electronic mail way too complicated under the guise of fighting spam." I'm aware that it won't get past every spam filter, but it gets past both Gmail's and my own provider (a smaller, non-American one), which is good enough for my own use. My claim was that sending electronic mail is a solved problem, because it is.
  - bigiain 7y ago
    > My claim was that sending electronic mail is a solved problem, because it is. Agreed. I mean, I can _probably_ remember enough to be able to send mail using just telnet without even needing to look anything up. (And that mail, without any mime parts or urls - it very very likely to not fall into any of the spam filtering on my inbound email accounts...)
    - greglindahl 7y ago
      In the modern era, lots of mailservers run software that looks for weirdnesses in incoming email. I don't know, but I'd suspect that the typing delays of using telnet to send mail are likely to trigger their defenses.
      - bigiain 7y ago
        SO I just tested. After working out that on macOS I now need to use nc -c instead of telnet... I _can_ send mail manually, but it _does_ end up in gmails's spam folder saying "Why is this message in spam? It is similar to messages that were identified as spam in the past." Iain-2:~ iain$ nc -c gmail-smtp-in.l.google.com 25 220 mx.google.com ESMTP k5si5682618pls.209 - gsmtp EHLO elided.org 250-mx.google.com at your service, [202.171.181.100] 250-SIZE 157286400 250-8BITMIME 250-STARTTLS 250-ENHANCEDSTATUSCODES 250-PIPELINING 250-CHUNKING 250 SMTPUTF8 MAIL FROM: <bigiain@elided.org> 250 2.1.0 OK k5si5682618pls.209 - gsmtp RCPT TO: <bigiain@gmail.com> 250 2.1.5 OK k5si5682618pls.209 - gsmtp DATA 354 Go ahead k5si5682618pls.209 - gsmtp Subject: Test From: bigiain@elided.org To: bigiain@gmail.com Date: Fri, 31 Jan 2020 13:07:00 +1100 Foo baz nah . 250 2.0.0 OK 1580436443 k5si5682618pls.209 - gsmtp QUIT 221 2.0.0 closing connection k5si5682618pls.209 - gsmtp Iain-2:~ iain$
cosmotic 7y ago
Most ISPs block SMTP traffic, which creates a pretty significant barrier to sending email. Even VPS services send through a secret gateways with all sorts of filters and controls on it.
gwittel 7y ago
Sending at scale is the problem. A provider like Mailgun (aka an ESP), is a huge target for abuse. Free and low priced tiers are tricky since they need to draw in customers, but then get heavily abused. tl;dr: Abuse ruins it for everyone. I work on the filtering side of the world so I see a lot of the challenges that even purportedly good actors face. Each provider does better or worse at controlling abuse. These things all have a cost: * IPs -- You need to send from a variety of IPs. They have warmup time before they can be safely used. One bad guy can burn that IP and everyone who shares. Most ESPs aren't going to have dedicated IPs for each (or those are for high tiers). * Dmarc/dkim/spf -- Passing validation can be hard since it requires your client to work with you. * Abuse reports -- Likely huge volume * Bounces -- They can count against the customer. How much is OK (e.g. AWS SES will cut you off if your bounce rate gets too high)? * Throwaway account abuse -- Huge issue, this threshold is a simple hammer (so now bad guys need ~10x more throwaway accounts). * Account takeover -- That good guy is now sending out crap. Now what? * Post-abuse cleanup -- Good luck going around and working with major providers to get yourself unblocked. Its a huge time sink. By using Mailgun, etc all of the above becomes their problem. On top of it they'll offer analytics, help crafting content that works in varying mail clients, etc. A small business type sender doesn't need much of this (other than maybe the technical help). But at some point, the scales tip.
- webbie917 7y ago
  ^ Couldn't agree more, especially "Abuse ruins it for everyone."
- quicksilver03 7y ago
  > IPs -- You need to send from a variety of IPs. They have warmup time before they can be safely used. One bad guy can burn that IP and everyone who shares. Most ESPs aren't going to have dedicated IPs for each (or those are for high tiers). I'm curious, is that specific to IPv4 or you see the same phenomenon with IPv6 where providers allocate /64 or /48 to customers? Or there simply isn't enough sending volume from IPv6 at the moment to warrant specific filtering?
  - gwittel 7y ago

NightlyDev 7y ago

I've sent a lot of mail in the last 14 years, and it's not hard. I spammed with invite mails to users friends back in 2008 and mail started to end up in spam for a while, but after switching to double opt-in for all mail and making unsubscribing easily available it went back to good. Add some periodic list cleaning of long time inactive subscribers and it works perfectly! It does get harder if you have to account for bad users, but if you're the only user then you can make sure everything is good. So, is it hard? That depends entirely on how wanted or unwanted the sent mail is.