5 ms·

Oof. So,I tried sending mails from our own domain and our mail server - but Google tagged us as spam. We migrated into Google hosting and it was fixed, now back

by edoceo 6y ago

Oof. So,I tried sending mails from our own domain and our mail server - but Google tagged us as spam. We migrated into Google hosting and it was fixed, now back to Spam. So we move to a provider like sendgrid, but then they have a hack. Damnit! When we hosted our own, we had DKIM, SPF, DMARC and such, clean IP and clean messages, we only sent account create and transaction notices - low volume, relevant messages to folks who opt-in. We had our little box setup right and we're polite actors but got tagged which killed our user experience (password resets went to Spam and folk called support) But it feels the "big-web" forced us into vulnerable providers for the spam game. We need more self hosted email and less centralized control.

smnplk 6y ago
`We need more self hosted email and less centralized control` We need to start over and build a truly open internet.
- Reelin 6y ago
  > We need to start over and build a truly open internet. I want to agree, but email is actually an example of an open and federated protocol achieving success. It's pretty much everything that could have been hoped for as far as openness and interoperability go at the protocol level. Unfortunately, balkanization still occurs at the social (ie network) level.
- dredmorbius 6y ago
  The open email Internet was buried in an avalanche of spam. Open alone is insufficient.
  - paulryanrogers 6y ago
    If email were architected to put the cost of storage on senders then I think the spam problem would have been more manageable. Instead it's so closely modeled on physical mail that recipients get overwhelmed quickly.
    - smabie 6y ago
      Even if email worked like that, I don't think it would solve the spam problem. Spam emails are rarely personalized and the same exact message is sent out to many different addresses. As such, the spammer would only have to store each unique message and the list of recipients associated with each unique message.
      - dredmorbius 6y ago
        Even with single copy + list, with a poll-and-request delivery model (sender polls recipient's system, which then requests message on some schedule): 1. Sender canot simply mass-cram messages to recipients, as many spam engines today do. (Qmail for a time was notorious for doing this). 2. There's good chance that any bulk spammer will accrue a negative reputation by the time most recipients bother to request messages. So they won't. 3. Spammer must sustain a stable online presence --- not be knocked offline or blocked by ISP or hosting provider. 4. Spammer has to track successful deliveries vs. messages that are complete. Upshot: the spammer's workfactor increases.
    - bsder 6y ago
      Hrm. I wonder if this is an actual use case for blockchain and proof of work. If you want to send me an email, you have to send me an SMTPCoin. If my user doesn't mark it as spam, I give you the coin back. If my user marks it as spam, I keep the coin.
      - jonas21 6y ago
        Preventing email spam was in fact the original use case for proof of work. https://en.wikipedia.org/wiki/Hashcash https://en.wikipedia.org/wiki/Hashcash
        _heimdall 6y ago
        This was the inspirstion for nano's use of proof of work too. They don't use PoW for the chain itself but do require every transaction to include a small proof of work to avoid spamming the network, almost identical to hashcash.
  - deleted 6y ago
    [deleted]
Silhouette 6y ago
We need more self hosted email and less centralized control. This is true, from both sides of the equation. The biggest single problem with email today, IME at least, is the number of major mail providers who are acting as gatekeepers for their users and doing a bad job of it. In particular, they block way too much legitimate mail, and often do it silently, so the sender is not even aware of the problem or able to contact them about getting it fixed. Then the sender of the missing mail gets the customer support requests about the missing password reset emails, or the complaints that someone didn't know they were still subscribed despite the receipt emails being sent for each payment, or... At this point, there really ought to be a blacklist for unreliable mail services on the receiving side analogous to the spam blacklists, so businesses can warn their users if given an address on a bad service and invite them to choose another.
jacquesm 6y ago
At this point in time I think I'd rather deal with the spammers than with Google, Microsoft etc. Unfortunately they've managed to lock up email to all intents and purposes, good luck reversing the situation to the point where they have to play nice.
TylerE 6y ago
To be honest, at this point I wish email would just die and get replaced with an actual modern protocol. It's a nightmare. Everything about it sucks.
- samat 6y ago
  Please don't. Anything modern would another lock-in. I can read decade old emails, switched multiple providers. Messages in those 'new protocols' are long unaccessible and protocols themselves are dead.
  - Spivak 6y ago
    At this point at least pay for delivery would give you some path to actually get to users.
    - zelly 6y ago
      Spammers would pay too, maybe even outbid the genuine users. They already invest a lot into getting past filters.
- smabie 6y ago
  Such as? How would it work?
  - TylerE 6y ago
    Not having trivially forged headers would be a start.
    - Spivak 6y ago
      Isn’t this basically what SPF, DKIM and SMTP over TLS get you? Sure you can try to forge messages but they should bounce immediately and not even reach the recipients inbox.
      - 6y ago
chromedev 6y ago
There are plenty of people successfully hosting their own email. If you're getting flagged as spam, then you need to take a few steps every once in a while but self-hosted email is still totally doable.
- foepys 6y ago
  As somebody who is running their own mail sever I'd love to agree with you but GMail, Hotmail, and all the other large email providers make it extremely hard to do so. If you only send a few emails per month, you are fine. When you host an email server for others, even if it's only your family and friends, it can quickly become a nightmare. And if I ran a business, I'd never do it myself. Landing on a blacklist of those large providers effectively shuts your email server down and it's way too easy to get blacklisted. There are so many techs around email now (DKIM, TLS, SPF, HELO testing), that coordinating them becomes a huge responsibility. Misconfiguring a single one can lead to rejection at the large providers that deal with billions of incoming emails a day, with hyper aggressive spam filters. The worst thing that can happen is that your emails were to be classified as spam but you didn't know it. A flat out rejection is relatively rare so you are often not notified that your configuration is incorrect. Getting off those blacklists is extremely hard. Especially if you don't land on GMail's blacklist but on those community driven ones that only allow your IP to be removed from it once. If you land on them a second time, you need a new IP and hope that this IP isn't already blacklisted. Many large providers are blacklisting entire ISP IP ranges due to past so waves. E.g. GMail is blocking the entire AWS EC2 range, I believe.
tialaramex 6y ago
Keep in mind that some users almost certainly believe your email is spam. Even if they're explicitly paying you to send them that email. For an end user "spam" is "email I don't want". "But" I hear you screaming, "That's their credit card bill, they even confirmed their email address and specifically opted into receiving it by email - that's not spam". It's unwanted, they don't have $850 to pay off the balance and they don't want to think about the interest charges, so it's spam as far as the user is concerned, some non-trivial fraction of users will click "Spam". You've probably seen people who have piles of unopened real world postal mail, they feel the same way about that. Unsolicited catalogue. Bank statement showing I am overdrawn. Nagging letter from my mother. All unwanted: Spam. Worse, some of these same users will anyway be angry that the important message they marked as spam is treated as spam. Why are facts true? They shouldn't be true if I don't want them to be. Is that irrational? Sure, that's humans for you. This is a losing proposition, and the end-user facing providers are aware of that, but "spam blocking" is something users demand from such providers even while being very angry that it doesn't do what they want. "Humans are irrational" is not actionable.
- DyslexicAtheist 6y ago
  > For an end user "spam" is "email I don't want". last time I checked (been a few years since I used Google) BigTech is perfectly able to distinguish a bad actor from an opt-in type of service. They even presented me with a little notification whenever I moved stuff into SPAM that didn't belong there asking "do you want Gmail to unsubscribe you from X" ... I can't exactly recall the text - but all their lofty claims about their "AI" should account for something? I agree with you that the _user_ is the biggest wildcard. But it's not that we should allow Google/Microsoft get way with this as an excuse. They are currently right in the middle of killing open protocols like imap in favor of their own flavored shit (probably something AMP like but for email). They do not deserve the benefit of a doubt. They do not even deserve to have their side of the argument heard at this stage.
  - Arnavion 6y ago
    GMail prompting you to unsubscribe is because the email carries a standard [1] header (List-Unsubscribe) and GMail detects it. They may also be using some smarts to detect unsubscribable emails that don't have the header, but in general it's because of the header. It doesn't have anything to do with distinguishing between spam and legitimate subscriptions. So if you mark a legitimate subscription as spam and don't unsubscribe when gmail prompts you to, it's likely that it'll get sent to your spam in the future, and perhaps contribute to the sender's spam score which affects other people's inboxes too. [1]: de-facto standard? I can't find any RFC for it.

by edoceo 6y ago

smnplk 6y ago

`We need more self hosted email and less centralized control` We need to start over and build a truly open internet.

Reelin 6y ago
> We need to start over and build a truly open internet. I want to agree, but email is actually an example of an open and federated protocol achieving success. It's pretty much everything that could have been hoped for as far as openness and interoperability go at the protocol level. Unfortunately, balkanization still occurs at the social (ie network) level.
dredmorbius 6y ago
The open email Internet was buried in an avalanche of spam. Open alone is insufficient.
- paulryanrogers 6y ago
  If email were architected to put the cost of storage on senders then I think the spam problem would have been more manageable. Instead it's so closely modeled on physical mail that recipients get overwhelmed quickly.
  - smabie 6y ago
    Even if email worked like that, I don't think it would solve the spam problem. Spam emails are rarely personalized and the same exact message is sent out to many different addresses. As such, the spammer would only have to store each unique message and the list of recipients associated with each unique message.
    - dredmorbius 6y ago
      Even with single copy + list, with a poll-and-request delivery model (sender polls recipient's system, which then requests message on some schedule): 1. Sender canot simply mass-cram messages to recipients, as many spam engines today do. (Qmail for a time was notorious for doing this). 2. There's good chance that any bulk spammer will accrue a negative reputation by the time most recipients bother to request messages. So they won't. 3. Spammer must sustain a stable online presence --- not be knocked offline or blocked by ISP or hosting provider. 4. Spammer has to track successful deliveries vs. messages that are complete. Upshot: the spammer's workfactor increases.
  - bsder 6y ago
    Hrm. I wonder if this is an actual use case for blockchain and proof of work. If you want to send me an email, you have to send me an SMTPCoin. If my user doesn't mark it as spam, I give you the coin back. If my user marks it as spam, I keep the coin.
    - jonas21 6y ago
      Preventing email spam was in fact the original use case for proof of work. https://en.wikipedia.org/wiki/Hashcash https://en.wikipedia.org/wiki/Hashcash
      - _heimdall 6y ago
        This was the inspirstion for nano's use of proof of work too. They don't use PoW for the chain itself but do require every transaction to include a small proof of work to avoid spamming the network, almost identical to hashcash.
- deleted 6y ago
  [deleted]

Silhouette 6y ago

We need more self hosted email and less centralized control. This is true, from both sides of the equation. The biggest single problem with email today, IME at least, is the number of major mail providers who are acting as gatekeepers for their users and doing a bad job of it. In particular, they block way too much legitimate mail, and often do it silently, so the sender is not even aware of the problem or able to contact them about getting it fixed. Then the sender of the missing mail gets the customer support requests about the missing password reset emails, or the complaints that someone didn't know they were still subscribed despite the receipt emails being sent for each payment, or... At this point, there really ought to be a blacklist for unreliable mail services on the receiving side analogous to the spam blacklists, so businesses can warn their users if given an address on a bad service and invite them to choose another.

jacquesm 6y ago

At this point in time I think I'd rather deal with the spammers than with Google, Microsoft etc. Unfortunately they've managed to lock up email to all intents and purposes, good luck reversing the situation to the point where they have to play nice.

TylerE 6y ago

To be honest, at this point I wish email would just die and get replaced with an actual modern protocol. It's a nightmare. Everything about it sucks.

samat 6y ago
Please don't. Anything modern would another lock-in. I can read decade old emails, switched multiple providers. Messages in those 'new protocols' are long unaccessible and protocols themselves are dead.
- Spivak 6y ago
  At this point at least pay for delivery would give you some path to actually get to users.
  - zelly 6y ago
    Spammers would pay too, maybe even outbid the genuine users. They already invest a lot into getting past filters.
smabie 6y ago
Such as? How would it work?
- TylerE 6y ago
  Not having trivially forged headers would be a start.
  - Spivak 6y ago
    Isn’t this basically what SPF, DKIM and SMTP over TLS get you? Sure you can try to forge messages but they should bounce immediately and not even reach the recipients inbox.
    - 6y ago

chromedev 6y ago

There are plenty of people successfully hosting their own email. If you're getting flagged as spam, then you need to take a few steps every once in a while but self-hosted email is still totally doable.

foepys 6y ago
As somebody who is running their own mail sever I'd love to agree with you but GMail, Hotmail, and all the other large email providers make it extremely hard to do so. If you only send a few emails per month, you are fine. When you host an email server for others, even if it's only your family and friends, it can quickly become a nightmare. And if I ran a business, I'd never do it myself. Landing on a blacklist of those large providers effectively shuts your email server down and it's way too easy to get blacklisted. There are so many techs around email now (DKIM, TLS, SPF, HELO testing), that coordinating them becomes a huge responsibility. Misconfiguring a single one can lead to rejection at the large providers that deal with billions of incoming emails a day, with hyper aggressive spam filters. The worst thing that can happen is that your emails were to be classified as spam but you didn't know it. A flat out rejection is relatively rare so you are often not notified that your configuration is incorrect. Getting off those blacklists is extremely hard. Especially if you don't land on GMail's blacklist but on those community driven ones that only allow your IP to be removed from it once. If you land on them a second time, you need a new IP and hope that this IP isn't already blacklisted. Many large providers are blacklisting entire ISP IP ranges due to past so waves. E.g. GMail is blocking the entire AWS EC2 range, I believe.

tialaramex 6y ago

Keep in mind that some users almost certainly believe your email is spam. Even if they're explicitly paying you to send them that email. For an end user "spam" is "email I don't want". "But" I hear you screaming, "That's their credit card bill, they even confirmed their email address and specifically opted into receiving it by email - that's not spam". It's unwanted, they don't have $850 to pay off the balance and they don't want to think about the interest charges, so it's spam as far as the user is concerned, some non-trivial fraction of users will click "Spam". You've probably seen people who have piles of unopened real world postal mail, they feel the same way about that. Unsolicited catalogue. Bank statement showing I am overdrawn. Nagging letter from my mother. All unwanted: Spam. Worse, some of these same users will anyway be angry that the important message they marked as spam is treated as spam. Why are facts true? They shouldn't be true if I don't want them to be. Is that irrational? Sure, that's humans for you. This is a losing proposition, and the end-user facing providers are aware of that, but "spam blocking" is something users demand from such providers even while being very angry that it doesn't do what they want. "Humans are irrational" is not actionable.

DyslexicAtheist 6y ago
> For an end user "spam" is "email I don't want". last time I checked (been a few years since I used Google) BigTech is perfectly able to distinguish a bad actor from an opt-in type of service. They even presented me with a little notification whenever I moved stuff into SPAM that didn't belong there asking "do you want Gmail to unsubscribe you from X" ... I can't exactly recall the text - but all their lofty claims about their "AI" should account for something? I agree with you that the _user_ is the biggest wildcard. But it's not that we should allow Google/Microsoft get way with this as an excuse. They are currently right in the middle of killing open protocols like imap in favor of their own flavored shit (probably something AMP like but for email). They do not deserve the benefit of a doubt. They do not even deserve to have their side of the argument heard at this stage.
- Arnavion 6y ago
  GMail prompting you to unsubscribe is because the email carries a standard [1] header (List-Unsubscribe) and GMail detects it. They may also be using some smarts to detect unsubscribable emails that don't have the header, but in general it's because of the header. It doesn't have anything to do with distinguishing between spam and legitimate subscriptions. So if you mark a legitimate subscription as spam and don't unsubscribe when gmail prompts you to, it's likely that it'll get sent to your spam in the future, and perhaps contribute to the sender's spam score which affects other people's inboxes too. [1]: de-facto standard? I can't find any RFC for it.