6 ms·

Urg. The thing is, I run my own personal email server. It's not used for spam ever. It's only personal email to family members, friends, and the odd reach out

Urg. The thing is, I run my own personal email server. It's not used for spam ever. It's only personal email to family members, friends, and the odd reach out to support. The volume of email is so small it's ridiculous. I might send up to 10 mail (1 receiver each) per month. I set it up with SPF and DKIM properly, triple checked. And yet, for whatever reason, sometimes my email end in spam. Generally when I send attachment (like pictures). But sometimes it's just completely arbitrary. How about this : for you it might be a "few false positive", because so much of the wanted mail traffic originates from google in the first place. But for the few of us that use mail as was intended, this becomes a huge problem. Having a mailing list doesn't necessarely mean the content is unwanted. I'm subscribed to HN replies for instance. I want those email. If it started going to spam, I might consider "hey it's a few false positive". But for the HN replies guys, it would be maddening to be unable to fix the problem because Google can't be arsed to have a support channel. From my POV, Google is way past "aggressive filtering" territory and way into "monopolistic behavior".

mikl 9y ago
Yeah, I’ve seen the same, to the point that I’ve actually considered getting a Gmail account for my business, because customers not receiving important e-mail is bad for business.
- mysterydip 9y ago
  And I think that's half their goal. Why fix something if it gets you more customers (direct or roundabout)?
  - mikl 9y ago
    Yeah, it certainly looks rather sinister.
acidburnNSA 9y ago
Ok so as much as I promote self-hosting email, you're right. I've also had trouble with mail originating on my server to gmail and only gmail accounts. I did recently discover that I had some configuration issues via mail-tester.com though (my rDNS didn't match the HELO of my server) so now that I'm getting a 10/10 on that, things seem a bit better. I definitely felt that gmail was being too aggressive on this too. I sent a bunch of detailed replies to middle/high schoolers working on science projects when they reached out through my public-education webpage and when they went to gmail I never got a reply but if they went to some other server, the conversations usually continued nicely. That was pretty frustrating.
nfriedly 9y ago
How are you subscribed to HN replies? I had something set up a year or two ago, but it stopped working.
- throwanem 9y ago
  http://www.hnreplies.com/ http://www.hnreplies.com/ works well for me.
  - nfriedly 9y ago
    Thanks!
thg 9y ago
If you haven't already, it can't hurt adding your sending domain to https://www.dnswl.org https://www.dnswl.org
- roblabla 9y ago
  Thx for the hint, I didn't know about this ! Just added my domain details to it, hopefully it helps a little.
mabbo 9y ago
> I set it up with SPF and DKIM properly, triple checked. And yet, for whatever reason, sometimes my email end in spam. It's probably your IP address. Sure, SPF and DKIM say you are allowed to send these emails, you are who you say you are, etc, but you also aren't a Sendgrid IP address (or someone similar) so you're treated as less trustworthy. I could be wrong, but I believe many email servers take these things into consideration (please let me know if I'm mistaken!)
- problems 9y ago
  This is fairly common - if you want a method that gets you around this at the cost of some privacy, you can use Mailgun or a similar service for free as your relay, which should pass most spam filters without issue.
  - JohnTHaller 9y ago
    Mailgun has similar issues though. We advise those signing up for accounts on our open source community site not to use Hotmail since mailgun emails tend to get binned by Hotmail. Not sorted to spam or bounced, straight up deleted.
- rtx 9y ago
  Is this anti NN.
  - ceejayoz 9y ago
    No.
    - jff 9y ago
      However, it's still Real Stupid.
      - ceejayoz 9y ago
        I'm pretty happy with the results of spam filtering based on sender reputation.
js2 9y ago
I had my personal domain hosted with Google. One of the reasons I moved away is that emails that I was sending to other Google Apps domains were bouncing as spam when the destination address was an alias. The bounce came from Google Groups, which is what Google used for handling aliases within Apps domains. e.g. https://pastebin.com/u48DAaLP https://pastebin.com/u48DAaLP That particular example was sent via SMTP, but I had the same problem when sending via the Gmail web interface, and it occurred sending to at least three different Google Apps domains. After I moved my domain off Google Apps (I switched to Fastmail for a variety of reasons, but that issue was the kicker), I was able to send to those same addresses without issue. In fairness to Google, I was on the Google Apps free tier at the time, so there was nowhere to go for support.
- massaman_yams 9y ago
  ARC was designed to fix that kind of edge case, and Gmail was one of the first to start signing and validating ARC within the last couple months - so this may no longer be a problem. http://arc-spec.org/ http://arc-spec.org/
  - js2 9y ago
    What's the edge case? That the destination address is an alias? That I was using a personal domain? That Google implements aliases using Google Groups? I never understood what was causing these bounces. The entire email chain occurred within Google's control, passed SPF and DKIM checks, my domain isn't on any blacklists nor used for spam that I know of, and isn't used for anything but personal emails. And I've had it registered for years. Nor was it the content of the message. I had a friend with another Google Apps domain setup an alias for testing and when I mailed that address it bounced the same way, regardless of the message body. Emails to other addresses in his domain were not marked as spam... it was only Groups aliases that bounced.
pja 9y ago
I set it up with SPF and DKIM properly, triple checked. Check your reverse DNS as well. It helps if it matches the HELO advertised by your mail server.
Bartweiss 9y ago
Whether by monopoly or sheer blindness, Google leans really heavily towards only letting massive institutions through. I know a few people whose professional work involves a lot of delivering explicitly-requested, non-spam emails. With most email providers, they don't go to spam because they aren't spam. If they get flagged as spam servers (usually right after spinning up an instance, when the confidence is low), they can contact someone and get it sorted out. With Gmail? They have extensive lists of tips and tricks to "sneak through" emails that their users specifically asked for. They 'prime' servers with extra-safe traffic to avoid getting frozen in the low-confidence early emails. And they still regularly run into problems. They just don't have the size to be trusted by Google, or to get business contacts to solve the problems that do arise. It's a pretty concerning state of affairs - Gmail will deliver anything sometimes, but it seems to be a very high bar (vastly higher than "not spam") to ensure you don't regularly drop messages.
- haroldp 9y ago
  This is exactly my experience sending legit commercial emails to gmail. We jump through every hoop, we implement every suggestion, and they drop our email on the floor every time. I have no problem with deliveries anywhere else. They DGAF. The idea that their spam filter is particularly good is ridiculous too. I get more spam to my gmail accounts than to my self hosted mail accounts with greylisting and spamassassin.
  - Bartweiss 9y ago
    I'm completely confused by Gmail's spam filter. We've already discussed its extreme tendency to false positives, but it's quite good about false negatives - no spam hits my inbox. Except... Google claims >90% of spam never even reaches your Spam box, only the ambiguous messages. So why do I get 10x the spam at Gmail as at any other account? Are they not actually good at detection? Are they just a high value target that gets hit more often? Or (my guess) are they trying to get clever? Google Doc's spellcheck is infamously bad because they're trying to use machine learning approaches without having a simple English dictionary as a fallback. Is Gmail trying to detect solely on email sender/content and totally abandoning approaches like greylisting?
benou 9y ago
Fully agree. I am in the same situation as you, and I even relay my emails through my ISP SMTP to avoid the issues such as 'residential IP marked as spammer'. And yet, Gmail will randomly classify simple personal emails. It is the only mail provider to act like this, I never had any issue with any other provider (ISPs, CSPs...) And to add insult to injury, they have this page: https://support.google.com/mail/answer/6227174?hl=en&ref_topic=6259779 https://support.google.com/mail/answer/6227174?hl=en&ref_top... but as I send too few emails, it is completely useless.
euyyn 9y ago
> Having a mailing list doesn't necessarely mean the content is unwanted. I'm subscribed to HN replies for instance. I want those email. If it started going to spam, I might consider "hey it's a few false positive". But for the HN replies guys, it would be maddening to be unable to fix the problem because Google can't be arsed to have a support channel. I'm not sure this argument is related to the rest of your post, but FWIW, GMail/Inbox puts emails from mail lists into their own "Forums" category, not in the spam folder.
inopinatus 9y ago
SPF & DKIM are only the start of the story. Use a non consumer IPv6 address, TLS, and DMARC. Have perfect DNS (including correct reverse lookups). Sign your zones. Monitor your IP and domain status with every reputation list on earth. Sign up for the postmaster tools at Google and Outlook. Running your own email ain't a hobbyist thing anymore, you gotta play like you're in the NBA. I'm doing all the above, my deliverability is solid. And pay attention to the changing landscape - for example, I will be implementing ARC real soon.
- CaptSpify 9y ago
  > Running your own email ain't a hobbyist thing anymore, you gotta play like you're in the NBA I'd disagree. I've been running my own just fine for years. Yeah, you have to put some work into it, but spf/dkim/dns have been suiting me just fine.
  - inopinatus 9y ago
    I think this is a matter of perspective. My response was to someone who, in my assessment of what they'd listed, had more of that work to put in. Note they were also talking of using email as a file transfer protocol, which you may not be doing. Also your years of operation may (as with my 17 years of spamless service) have built a very positive reputation for domains and addresses.
- roblabla 9y ago
  I mean, DMARC, ARC and TLS make perfect sense (though I have none setup because DMARC is nuclear, I didn't know about ARC, and TLS is just up for the submission port for me). But signed DNS ? What does that protect against, malicious registrar ? And the postmaster tools don't work for me on google because the volume of email I send is too low. So there's that. On a sidenote : It's about time something like ARC comes out. If I understand it correctly, it should fix many setups that involve mail forwarding. That'd be a godsend for me, I'll probably set it up soon enough.
  - inopinatus 9y ago

mikl 9y ago

Yeah, I’ve seen the same, to the point that I’ve actually considered getting a Gmail account for my business, because customers not receiving important e-mail is bad for business.

mysterydip 9y ago
And I think that's half their goal. Why fix something if it gets you more customers (direct or roundabout)?
- mikl 9y ago
  Yeah, it certainly looks rather sinister.

acidburnNSA 9y ago

Ok so as much as I promote self-hosting email, you're right. I've also had trouble with mail originating on my server to gmail and only gmail accounts. I did recently discover that I had some configuration issues via mail-tester.com though (my rDNS didn't match the HELO of my server) so now that I'm getting a 10/10 on that, things seem a bit better. I definitely felt that gmail was being too aggressive on this too. I sent a bunch of detailed replies to middle/high schoolers working on science projects when they reached out through my public-education webpage and when they went to gmail I never got a reply but if they went to some other server, the conversations usually continued nicely. That was pretty frustrating.

nfriedly 9y ago

How are you subscribed to HN replies? I had something set up a year or two ago, but it stopped working.

throwanem 9y ago
http://www.hnreplies.com/ http://www.hnreplies.com/ works well for me.
- nfriedly 9y ago
  Thanks!

thg 9y ago

If you haven't already, it can't hurt adding your sending domain to https://www.dnswl.org https://www.dnswl.org

roblabla 9y ago
Thx for the hint, I didn't know about this ! Just added my domain details to it, hopefully it helps a little.

mabbo 9y ago

> I set it up with SPF and DKIM properly, triple checked. And yet, for whatever reason, sometimes my email end in spam. It's probably your IP address. Sure, SPF and DKIM say you are allowed to send these emails, you are who you say you are, etc, but you also aren't a Sendgrid IP address (or someone similar) so you're treated as less trustworthy. I could be wrong, but I believe many email servers take these things into consideration (please let me know if I'm mistaken!)

problems 9y ago
This is fairly common - if you want a method that gets you around this at the cost of some privacy, you can use Mailgun or a similar service for free as your relay, which should pass most spam filters without issue.
- JohnTHaller 9y ago
  Mailgun has similar issues though. We advise those signing up for accounts on our open source community site not to use Hotmail since mailgun emails tend to get binned by Hotmail. Not sorted to spam or bounced, straight up deleted.
rtx 9y ago
Is this anti NN.
- ceejayoz 9y ago
  No.
  - jff 9y ago
    However, it's still Real Stupid.
    - ceejayoz 9y ago
      I'm pretty happy with the results of spam filtering based on sender reputation.

js2 9y ago

I had my personal domain hosted with Google. One of the reasons I moved away is that emails that I was sending to other Google Apps domains were bouncing as spam when the destination address was an alias. The bounce came from Google Groups, which is what Google used for handling aliases within Apps domains. e.g. https://pastebin.com/u48DAaLP https://pastebin.com/u48DAaLP That particular example was sent via SMTP, but I had the same problem when sending via the Gmail web interface, and it occurred sending to at least three different Google Apps domains. After I moved my domain off Google Apps (I switched to Fastmail for a variety of reasons, but that issue was the kicker), I was able to send to those same addresses without issue. In fairness to Google, I was on the Google Apps free tier at the time, so there was nowhere to go for support.

massaman_yams 9y ago
ARC was designed to fix that kind of edge case, and Gmail was one of the first to start signing and validating ARC within the last couple months - so this may no longer be a problem. http://arc-spec.org/ http://arc-spec.org/
- js2 9y ago
  What's the edge case? That the destination address is an alias? That I was using a personal domain? That Google implements aliases using Google Groups? I never understood what was causing these bounces. The entire email chain occurred within Google's control, passed SPF and DKIM checks, my domain isn't on any blacklists nor used for spam that I know of, and isn't used for anything but personal emails. And I've had it registered for years. Nor was it the content of the message. I had a friend with another Google Apps domain setup an alias for testing and when I mailed that address it bounced the same way, regardless of the message body. Emails to other addresses in his domain were not marked as spam... it was only Groups aliases that bounced.

pja 9y ago

I set it up with SPF and DKIM properly, triple checked. Check your reverse DNS as well. It helps if it matches the HELO advertised by your mail server.

Bartweiss 9y ago

Whether by monopoly or sheer blindness, Google leans really heavily towards only letting massive institutions through. I know a few people whose professional work involves a lot of delivering explicitly-requested, non-spam emails. With most email providers, they don't go to spam because they aren't spam. If they get flagged as spam servers (usually right after spinning up an instance, when the confidence is low), they can contact someone and get it sorted out. With Gmail? They have extensive lists of tips and tricks to "sneak through" emails that their users specifically asked for. They 'prime' servers with extra-safe traffic to avoid getting frozen in the low-confidence early emails. And they still regularly run into problems. They just don't have the size to be trusted by Google, or to get business contacts to solve the problems that do arise. It's a pretty concerning state of affairs - Gmail will deliver anything sometimes, but it seems to be a very high bar (vastly higher than "not spam") to ensure you don't regularly drop messages.

haroldp 9y ago
This is exactly my experience sending legit commercial emails to gmail. We jump through every hoop, we implement every suggestion, and they drop our email on the floor every time. I have no problem with deliveries anywhere else. They DGAF. The idea that their spam filter is particularly good is ridiculous too. I get more spam to my gmail accounts than to my self hosted mail accounts with greylisting and spamassassin.
- Bartweiss 9y ago
  I'm completely confused by Gmail's spam filter. We've already discussed its extreme tendency to false positives, but it's quite good about false negatives - no spam hits my inbox. Except... Google claims >90% of spam never even reaches your Spam box, only the ambiguous messages. So why do I get 10x the spam at Gmail as at any other account? Are they not actually good at detection? Are they just a high value target that gets hit more often? Or (my guess) are they trying to get clever? Google Doc's spellcheck is infamously bad because they're trying to use machine learning approaches without having a simple English dictionary as a fallback. Is Gmail trying to detect solely on email sender/content and totally abandoning approaches like greylisting?

benou 9y ago

Fully agree. I am in the same situation as you, and I even relay my emails through my ISP SMTP to avoid the issues such as 'residential IP marked as spammer'. And yet, Gmail will randomly classify simple personal emails. It is the only mail provider to act like this, I never had any issue with any other provider (ISPs, CSPs...) And to add insult to injury, they have this page: https://support.google.com/mail/answer/6227174?hl=en&ref_topic=6259779 https://support.google.com/mail/answer/6227174?hl=en&ref_top... but as I send too few emails, it is completely useless.

euyyn 9y ago

> Having a mailing list doesn't necessarely mean the content is unwanted. I'm subscribed to HN replies for instance. I want those email. If it started going to spam, I might consider "hey it's a few false positive". But for the HN replies guys, it would be maddening to be unable to fix the problem because Google can't be arsed to have a support channel. I'm not sure this argument is related to the rest of your post, but FWIW, GMail/Inbox puts emails from mail lists into their own "Forums" category, not in the spam folder.

inopinatus 9y ago

SPF & DKIM are only the start of the story. Use a non consumer IPv6 address, TLS, and DMARC. Have perfect DNS (including correct reverse lookups). Sign your zones. Monitor your IP and domain status with every reputation list on earth. Sign up for the postmaster tools at Google and Outlook. Running your own email ain't a hobbyist thing anymore, you gotta play like you're in the NBA. I'm doing all the above, my deliverability is solid. And pay attention to the changing landscape - for example, I will be implementing ARC real soon.

CaptSpify 9y ago
> Running your own email ain't a hobbyist thing anymore, you gotta play like you're in the NBA I'd disagree. I've been running my own just fine for years. Yeah, you have to put some work into it, but spf/dkim/dns have been suiting me just fine.
- inopinatus 9y ago
  I think this is a matter of perspective. My response was to someone who, in my assessment of what they'd listed, had more of that work to put in. Note they were also talking of using email as a file transfer protocol, which you may not be doing. Also your years of operation may (as with my 17 years of spamless service) have built a very positive reputation for domains and addresses.
roblabla 9y ago
I mean, DMARC, ARC and TLS make perfect sense (though I have none setup because DMARC is nuclear, I didn't know about ARC, and TLS is just up for the submission port for me). But signed DNS ? What does that protect against, malicious registrar ? And the postmaster tools don't work for me on google because the volume of email I send is too low. So there's that. On a sidenote : It's about time something like ARC comes out. If I understand it correctly, it should fix many setups that involve mail forwarding. That'd be a godsend for me, I'll probably set it up soon enough.
- inopinatus 9y ago