9 ms·

I appreciate the response. From someone who has operated a mail server since 2002, it comes across that Gmail does not care about cooperating with small but leg

by dingus 7y ago

I appreciate the response. From someone who has operated a mail server since 2002, it comes across that Gmail does not care about cooperating with small but legitimate mail servers. SPF, DKIM, reverse DNS, no blacklists, no open relay, longtime ownership of IPs, etc etc. Using various mail testers returns a 10/10 deliverability score. And yet, messages sent to Gmail always go into the spam folder, or are never delivered at all. These are everyday regular messages, I have never used mailing lists or sent bulk automated messages. The issue is, there is no recourse, no fix, no acknowledgement of the problem with false positives. There is no tool available to me to understand or correct the "problem". Hint: this comes across as Gmail not giving a shit. Gmail has a responsibility to be more accountable, even if these problems are unintentional, because Gmail is such an enormous node in a federated network. > If we lose an open, healthy ecosystem with many providers, we'll destroy the base we stand on. Correct. Gmail is contributing to the erosion of email reliability. Please course correct.

thaumasiotes 7y ago
I've had gmail send the following to the spam folder: - Legitimate class action notices related to Amazon purchases. - Email coming from addresses to which I had already sent email. (!) - Email from my landlord. - Email coming from Google itself. Based on the contents of my spam folder, which I have to check fairly often because of the extreme overaggressiveness, I would be vastly better off if nothing ever got filtered at all. [1] >> There are going to be false positives, we will make mistakes, but we certainly care a lot about fixing issues like this when we hear about them. This doesn't sound honest, or at least not complete. People have been complaining about this for years. I have personally been complaining about this for years. The loss of obviously legitimate email is completely outrageous. It doesn't look intentional (look at that fourth category!), but it certainly doesn't look like anyone is trying to address the problem. [1] Yes, if spam filtering was disabled, more spam might get sent.
- bryanrasmussen 7y ago
  >Legitimate class action notices related to Amazon purchases. You'd think Google had an incentive to get that delivered.
- mintplant 7y ago
  > - Email coming from Google itself. Yep. I missed an invitation to a Google-hosted event at a conference I attended because the email (from an @google.com address, no less) got caught in Gmail's spam filter.
  - zwkrt 7y ago
    This is a problem with ML approaches, right? Instead of water boiling at "100C" it boils at "99.98C +- 0.04C". Normally this is ok, but sometimes it isn't!
    - yeahitslikethat 7y ago
      If something happens a trillion times a day with .00001 error rate, then 50 MILLION things went wrong, affecting, say 50,000 people. 50 of them blast twitter and it seems the world is collapsing. The world is resilient.
      - thaumasiotes 7y ago
        I don't think an average of 1,000 mistakes per person-affected-by-mistakes is realistic here. I'd bet it's closer to 1.
        deleted 7y ago
        [deleted]
      - jodrellblank 7y ago
        If there are 50,000 people and 50,000 people experience problems, that's bad. If there are 5 million people and 50,000 experience problems, that's fine? Isaac Asimov's comments about world population increase involved something about this; the more people there are, the more each individual is dehumanised and rendered irrelevant (my paraphrasing).
        yeahitslikethat 7y ago
- arpa 7y ago
  You know what's really funny? That even with that overly agressive spam filter, once in a while (once a quarter maybe), it somehow manages to miss obvious BuY@@NiGERiAn@@Vi@gRa-Cia1is type of emails... which, by the way, my morally outdated spamassasin marks as spam.
  - AceJohnny2 7y ago
    "morally outdated" ?
    - SXX 7y ago
      I would make an educated guess that "arpa" is from EX-USSR and their native language is Russian. It's direct translation of idiom "морально устаревший" which literally means something is: * Available for decades. * Far from being top notch technology. * Sometimes of course it's literally mean outdated. Like if you run older CentOS or Debian with decade-old packages. So it's doesn't mean SpamAssassin is bad, but it's very far from state-of-the-art ML technologies that Google might have.
      - blago 7y ago
        Bulgarian has that too: "Морално остарял"
- ljm 7y ago
  Before I switched to fastmail, gmail would not believe me when I marked an email as not-spam. They would still get flagged. All I wanted was updates from one band. That didn’t stop me from repeatedly getting junk from every other record label my email was sold to. It was an endless procession of shit I never subscribed to.
  - StudentStuff 7y ago
    "Inboxing" with Gmail & Outlook is a baroque, hellish process for mailservers. When either provider decides your small email server is sending spam (eg: sending an email with an attachment, or any kind of form email like a daily report) you won't get through to user inboxes, and instead you'll be routed to spam, or for Outlook.com hosted addresses they will accept mail from your server and send it to /dev/null. Gmail's process is bad, but Microsoft has decided to accept emails and throw them away (which is ridiculous).
    - chupa-chups 7y ago
      At my workplace we have two email addresses, one from google mail (gmail for business) and one from MS Exchange. I never had a problem with gmail for business regarding spam. I regularly receive mails from smaller businesses (some of them hosting their own mail server) and never had a complaint from anyone yet. Since i can also be contacted via phone i'd know. On the other hand, MS Exchange constantly delivers obvious spam mails and (quite seldomly, but still) swallows legitimate mail. Anectodal, i know. And disclaimer: the behavior depicted in the article is as bad as it gets, if everything is as described.
    - int_19h 7y ago
      As far as I know, Gmail also does this sort of black hole spam filtering - the stuff that you see in "Spam" isn't everything.
- cpeterso 7y ago
  Your comment motivated me to check my Gmail spam folder and 10% (3 out of 30) were false positives, two of which were pretty important. :(
  - ModernMech 7y ago
    Me too. I found an invitation for an on-site job interview I would have 100% missed if they had not called me as well.
    - influx 7y ago
      Likewise I found they filtered an e-mail from my bank about my car loan.
  - wingworks 7y ago
    I've recently started checking quite regularly my spam folder as I've noticed more and more legit emails ending up there. One of them being support emails from TradeMe (one of New Zealand's biggest sites), keep getting put into spam, even after multiple "mark as not spam", along with some other kinda important emails from TradeMe. I've had to put in manual filters to force an email from TM to skip the spam.
    - PenguinCoder 7y ago
      And yet, in my own none Gmail hosted email (fastmail), I currently have 2/55 false positive spam emails. I very rarely ever check it. To note I usually get one actual spam (non newsletter blog spam) to my actual inbox, a month.
    - mr_toad 7y ago
      TradeMe, like eBay and Banks is a popular target of phishing emails.
  - baudehlo 7y ago
    I recently missed an email from the UK government regarding a passport application, it went to Spam. Talk about missing the mark!
- type0 7y ago
  About accountability, this gmail thing vaguely reminds me about "The Drop" situation on the mobile phone network https://www.youtube.com/watch?v=pCOCKS5AJI8 https://www.youtube.com/watch?v=pCOCKS5AJI8
- tlogan 7y ago
  And all parents with kids in SF public schools should note that all emails from noreply@sfusd.edu (bulletins, etc.) are marked as spam.
- preinheimer 7y ago
  > - Email coming from Google itself. If email coming from google itself got special treatment there would be masses with pitchforks complaining about that. You've made a lot of good points, but I don't think that's one of them.
  - pfranz 7y ago
    I don't think that has anything to do with Google needing special treatment. It shows that the company who made the rules and has every privilege to follow them isn't able to.
    - preinheimer 7y ago
      I think that's a fair interpretation and a good point.
    - jrnichols 7y ago
      Valid point. They've become so large and synonymous with email that many people I encounter are actually unaware that there is other email besides "Gmail." It feels like Google no longer has any incentive to follow the rules, and they feel that they are going to be the ones to make the new rules. The rest of us end up having to implement workarounds.
  - thaumasiotes 7y ago
    > You've made a lot of good points, but I don't think that's one of them. I disagree. I think in the absence of that point, it would have been hard to say this: > It doesn't look intentional (look at that fourth category!) But also, I think special treatment for trusted actors is a completely appropriate way to handle email delivery, and I also think it's appropriate for gmail to trust themselves to be sending legitimate mail. Blocking their own email makes them look totally incompetent. They absolutely should whitelist themselves. And they should have a way for you to be whitelisted too, if you want to send email.
    - preinheimer 7y ago
      If they whitelisted the major players in the space, I'd worry that the problem would get _much_ worse for everyone else.
- sandworm101 7y ago
  >> I would be vastly better off if nothing ever got filtered at all. There are layers of filtering beyond what appears in your spam folder, layers that block obvious spam long before it gets anywhere near your account. If every email ever sent to your address wound up in your spam folder you'd beg for filtering.
  - thaumasiotes 7y ago
    I'm prepared to believe this. But it's not a defense of gmail's policies -- this suggests that in fact nothing would be lost if gmail eliminated the spam folder and delivered everything that would have gone there to your inbox instead. So why are they doing this?
  - badsectoracula 7y ago
    Before switching to shared hosting from my VPS (one reason being that i didn't want to bother with email maintenance and didn't want to have a separate service just for email), i had my mail with (badly configured) Spamassassin that wouldn't delete mail, just add a "SPAM" prefix in the topic. My mail isn't exactly commonly known, but still is one i have for years and was made public thanks to me releasing an Android app once (after which, spam increased dramatically). Even after running it for years, Spamassassin never marked a legitimate mail as spam, so i'm pretty sure that if i wasn't too lazy to configure it to move it to a spam folder, it'd work fine. Stuff did pass through it (at a ratio of one every four or so) but i was fine with deleting those. What i'm trying to say is that from personal experience, i'd be fine with a spam filter that errs on the side of not marking stuff for spam and me deleting whatever goes through manually. Having to see a bit of spam mail is small cost for losing mail i'm actually interested in.
    - deleted 7y ago
      [deleted]
  - vbezhenar 7y ago
    I'm using my e-mail pretty much everywhere (me@vbezhenar.com you have it in clear text, every bot will crawl it now, not for the first time, though). I'm too lazy to setup spamassasin yet, so I'm getting a notification for every spam message I got. The only thing that I'm trying to do is to click "unsubscribe" even from obviously spam mails (may be it'll make more harm than good, not sure). So I'm getting around 20-30 spam e-mails per day. I don't think that it's THAT bad. I'm spending may be a minute every day to delete it. And I'm sure that with absolutely minimal spam filtering I would achieve almost perfect filtering.
- mathattack 7y ago
  I had a communication from a Google recruiter go to my Gmail spam folder. But it still beats no filtering.
- eq_sd_ 7y ago
  I've also had email from a Google recruiter go to the spam box...
- m463 7y ago
  I think the SPAM folder is a separate issue (although important) The article discusses mail not being accepted by google/gmail in the first place.
  - thaumasiotes 7y ago
    Fair comment. But that issue is pretty simple: there is no good reason for them to fail to deliver email under any circumstances. (This has happened to me too -- someone tried to email my gmail account and gmail completely refused to deliver it. It was pretty embarrassing.) Messages they think you won't want to receive are what the spam folder is for.
    - m463 7y ago
      "fail to deliver email under any circumstances." The amount of spam that would be delivered if they didn't discriminate AT ALL is enormous. They have to read all this feedback and discriminate better.
    - winkeltripel 7y ago
      > there is no good reason for them to fail to deliver email under any circumstances Yes there is. They don't want to carry traffic from anybody from the major email blacklists. If a mail server is on a real, very transparently-managed blacklist, no large provider should be accepting their smtp traffic.
      - acct1771 7y ago
        Yet all the people here trying to administrate servers from residential and VPS blocks of IPs are telling you they're caught up in this list you're lauding as all-knowing and safe.......
- will4274 7y ago
  I've also had email from Google recruiters (@google.com email addresses) go to spam. I considered this a high level indicator that Google spam filtering is incompetent, not malicious - if it was malicious, they would at least be able to get their corporate emails through.
- ma2rten 7y ago
  - Email coming from addresses to which I had already sent email. (!) That is understandable. It is hard to validate if an email is authentic. SMTP has no authentication built in. Gmail can't just blindly accept all emails from addresses that you have already sent an email to. EDIT: Look at this example: https://en.wikipedia.org/wiki/Simple_Mail_Transfer_Protocol#SMTP_transport_example https://en.wikipedia.org/wiki/Simple_Mail_Transfer_Protocol#... Anyone can connect to relay.example.com and pretend to be bob@example.com.
  - bdhess 7y ago
    Huh? If it's in the Spam folder, they do accept it.
    - ma2rten 7y ago
      I meant accept as not spam.
  - muppetman 7y ago
    Not if you use SPF/DKIM/DMARC they can't, that's the whole point of those various additions. All those "I hacked your email and send you a message from you account" I don't get, because I have a DMARC policy that says if you don't pass SPF/DKIM then you get rejected. So try as the spammer might to connect to my mailserver and pretend to be me, they can't, because my mailserver sees they're not authenticated, and the mailserver they're sending from isn't in my SPF records, isn't signing the message with my DKIM key and therefore it gets rejected at the SMTP level.
  - deno 7y ago
    That’s what SPF[1] is for, and optionally DMARC[2]. [1] https://en.wikipedia.org/wiki/Sender_Policy_Framework https://en.wikipedia.org/wiki/Sender_Policy_Framework [2] https://en.wikipedia.org/wiki/DMARC https://en.wikipedia.org/wiki/DMARC
- Fogest 7y ago
  I just cleaned out a little over 100 emails in my Gmail spam filter yesterday. About 80% of what was in there were emails from YouTube giving me notifications of new videos people have uploaded that I am subscribed to. These emails never used to go to spam, but slowly over time more and more of them would end up in spam. It's at the point now where almost all of them from YouTube go to the spam folder. It doesn't make any sense since they are emails from Google, they are emails I even have a filter applied to so that a label is applied to them. Yes I can adjust the filter and choose "never send to spam" but the messages will still show a warning on them saying "This message was not sent to spam because of a filter you have applied". Sure false positives makes sense, but I don't get how the majority of what is in my spam folder would be emails sent by Google.
  - Faark 7y ago
    > It's at the point now where almost all of them from YouTube go to the spam folder. It doesn't make any sense since they are emails from Google It makes a lot of sense... people use the spam button as a lazy man's unsubscribe. Youtube adding the bell button, making mail opt in is probably a response to that. I actually give Google a lot of cred for not simply white-listing its own domains. Though spammers would probably find ways to abuse it and make them look bad anyway.
    - ctack 7y ago
      > people use the spam button as a lazy man's unsubscribe This is the small mail server crux right here. If you’re a small mail server and a few of your emails have been spam binned instead of unsubscribed, it would likely lead to your whole server getting shit canned.
- knorker 7y ago
  > - Email coming from Google itself. To be fair, sometimes Google does send spam.
tankenmate 7y ago
Normally I wouldn't +1 a message because it doesn't really seem to add much to the message. But in this case I will (even though it might earn me ire from some). If I could highlight the parent in bold I would. I've run mail servers since the mid 90's and the experience above exactly mirrors my experience. And my takeaway is the same, GMail just doesn't seem to care. I understand that GMail can't give away too much info on why a particular email ghosted (opsec and all that) but GMail doesn't need to do a better job of explaining to postmasters what good actors can do to avoid being ghosted, and more importantly have that reflect actual real life experience.
- russh 7y ago
  Add me to the "I've run mail servers since the mid 90's and the experience above exactly mirrors my experience." list.
  - kreetx 7y ago
    I think all of you should start some kind of a movement so that this could be heared better at GMail's end.
    - dhess 7y ago
      That's a great idea. I've run my own mail servers since 1999. I'd happily join a union of independent mail operators with the purpose of lobbying Google, Microsoft, and Yahoo to treat us as first-class citizens.
      - walterbell 7y ago
        Start a mailing list? Or is there an existing one for this topic?
        kreetx 7y ago
        I think you should have a web page where you list your issues, show how you've done all the normal things to be considered a well-behaved mail domain. And then perhaps report your experiences with iinteracting with gmail support on the matter.
_eht 7y ago
I will chime and validate this experience. You will not fully understand or realize the hostility google has towards open email unless you have working experience with email protocols. This is by design. It is my belief that this is intentional and I would love to be corrected if it’s not.
- jolmg 7y ago
  Reminds me of this email from Linus Torvalds about a Gmail bug that was affecting git: https://public-inbox.org/git/CAHk-=whP1stFZNAaJiMi5eZ9rj0MRt20Y_yHVczZPH+O01d+sA@mail.gmail.com/ https://public-inbox.org/git/CAHk-=whP1stFZNAaJiMi5eZ9rj0MRt...
lossolo 7y ago
I have similar experience and of course if you send exactly the same message but from gmail account, then it's not a spam for gmail anymore..
deleted 7y ago
[deleted]
xtiansimon 7y ago
Just to be clear, your comments are refering to problems with Gmail's Spam filtering. Isn't that a different issue than what the OP is saying--that his mailserver is being blocked? In other words, emails from his mail server wouldn't even make it to Spam folder.
- fgonzag 7y ago
  Gmail spam filtering and it's mail server are two sides of the same coin. The mail server itself is blocking believed spam messages as a first layer, then the spam filter acts a second layer that marks messages as spam. I guess they don't want to save messages that they are completely sure are spam, the problem seems to be their false positive rate or simply their approach against small operators.
deleted 7y ago
[deleted]
fpierfed 7y ago
This is totally my experience as well. It got so bad that I gave up operating my email server. Anyway, Gmail is getting some heat in this thread and rightfully so. We should however not forget that Microsoft and Yahoo are just as bad if not worse in this respect.
- GFischer 7y ago
  I´ve unfortunately searched for some customer-friendly e-mail provider and people want Google. I had set up some commercial accounts on Zoho (my own e-mail is on Zoho) but they asked me to migrate to Google, even though it was more expensive.
andrewprock 7y ago
Just checked my junk folder. 9 out of 10 threads marked as spam were false negatives. That's a 90% failure rate.
tty2300 7y ago
I have everything set up correct and gmail even accepts 90% of my emails but every now and then it randomly marks one of my emails as spam. A gmail user emailed me first and I replied with a text email no links and that gets marked as spam.
emmelaich 7y ago
> There is no tool available to me to understand I _think_ this is because G doesn't want people gaming the system. > or correct the "problem". I regularly fish mail from my spam bin. From mailing lists, and other important stuff -- and indeed Google's own mail! One thing you can do is to get people to add you to their contacts list. It's a hard problem. There is one "solution" you probably do not want. Have Google (and other companies) give their imprimatur to certain mail senders.
slacka 7y ago
First of all, if you check this site: https://toolbox.googleapps.com/apps/checkmx/check?domain=tablix.org https://toolbox.googleapps.com/apps/checkmx/check?domain=tab... It is not to be set up correctly. Now in your case, you failed to mention DMARC records. ALL of the big cloud email services have required that for years. Your SMTP server checklist is straight out of 2005. This entire thread reminds me of the guy who pulled out of a desk drawer an Analog Startac cell phone and blamed AT&T for it not working. Email has moved on. It not Google's fault that people are configuring SMTP servers like it 2005.
- mirimir 7y ago
  What does this mean? > error SPF must allow Google servers to send mail on behalf of your domain. So you must let Google spoof your domain? That seems crazy. ProtonMail has the same error: https://toolbox.googleapps.com/apps/checkmx/check?domain=protonmail.com&dkim_selector= https://toolbox.googleapps.com/apps/checkmx/check?domain=pro...
  - matrixagent 7y ago
    He's wrong about what this tool provides – namely information for people who want to use GSuite for their email. In that context it makes sense that you have to allow Google "spoofing" your domain, of course. But it has nothing to do with what is being discussed here.
    - mirimir 7y ago
      Ah. Thanks. It makes sense now. I didn't read carefully enough.
- roca 7y ago
  This tool checks that you have set up your domain correctly for use with Google Apps. For domains that are not using Google Apps, its results are meaningless.
- zlynx 7y ago
  It turns out DMARC isn't actually required. Having it might help your acceptance score, but gmail and outlook don't actually require it. I know because I haven't set up DMARC for my email domain, and my email still gets accepted by everyone. I believe it helps that I accept on TLS with a valid certificate, and I used Google's postmaster site verification.
craz8 7y ago
I sent mail from my personal gmail to my company email (different user) for expenses Since money was involved, I paid attention and followed up Our finance person found many emails in Spam that were important and should not be there On the other hand, my personal Spam folder is certainly full of crap I never want to see. But now I don’t trust the system, so I have to scan them anyway
travbrack 7y ago
Datapoint: Ive run a small exchange server with fewer than 500 users for 5 years and have never had an issue delivering to Gmail except a couple times when we had legit issues on our end.
wildduck 7y ago
Have you looked into WildDuck.email? I find it very nicely made for people who wants to run their own email. Still in Beta. FYI, I'm just a js developer not related to them.
flurdy 7y ago
I also have a 15+ years old set of mail servers and I host mail domains for 10ish friends and family. In fact, I once wrote how to set this up (http://flurdy.com/docs/postfix http://flurdy.com/docs/postfix) I no longer use Gmail myself but half of my users relay some aliases to their main Gmail account. No problems with that, except my servers continuously get rate limited by Google: Our system has detected an unusual rate of 421-4.7.0 unsolicited mail originating from your IP address. To protect our 421-4.7.0 users from spam, mail sent from your IP address has been temporarily 421-4.7.0 rate limited. Please visit 421-4.7.0 https://support.google.com/mail/?p=UnsolicitedRateLimitError to 421 4.7.0 review our Bulk Email Senders Guidelines. I have the normal anti-spam features set up (SPF, DKIM, DMARC, Postgrey, Spamassassin, etc) so most spam gets rejected but not all. But several of my users have very common English names as their aliases that are already guessed and added to many spam databases. This leads to family phoning me all the time to say my server is broken when it is just Google throttling everyone's emails. I frequently have to check Postfix queues and clear some obvious spam or just pointless Facebook update emails which seems to be the majority... (Yes your email admin can read your email...) The only way to improve is to constantly remind my family to not sign up to all crap, use not so common aliases, and try to keep tightening my anti-spam configurations. But we really are not talking about a lot of emails. Probably less than a hundred per day spread across 5-ish end accounts of which 95% is probably legit. Yet Google is treating me as some totally open relay. (╯°□°)╯︵ ┻━┻
- flurdy 7y ago
  I do understand Google's challenge with handling probably the largest amount of email traffic in the world, and then the most spam in the world, and that it is a continuously moving target. But it does feel like they are treating nearly all minor relays as spam relays. We can not block 100% of spam before we relay onwards to Gmail as that would mean too many false positives emails get blocked, but most try to block as much as possible. In my case, my servers probably block 99% of spam, but some will get relayed, and most of those ends up being handled by Gmail's even better Bayes scoring and filtered to end users' spam folders. I don't know what the automatic threshold limit is to be on their naughty list, but it must be very low, as in double digits per day.
hakfoo 7y ago
A few years ago, I was dealing with a few small business sites which were self-hosted and always had deliverability problems with the large mail services (mostly, back in the day, AOL and Yahoo). Large providers tended to have a world view dividing all senders into two categories: 1) Bulk senders who are clearly mailing the same spam to a list of a billion addresses 2) Non-commercial individual hosts which should be sending five messages a day or less in total. It felt like there was a huge missing third category for transactional emailers nobody wanted to acknowledge. They are probably difficult to score fairly. A hundred "Order details" emails are going to have the same level of randomness/templatedness as the old Viagra spam which had a random block of Project Gutenberg text pasted at the end to trip up filter math. You're not going to have a clear history of "this address bounced twice, let's stop sending newsletters" when most of your messages are to first time customers or once-every-few-years return ones. A lot of the messages will look generic because they use default shopping cart templates. To the extent they provided sender guidance, it was focused around use case 1) -- sign up for feedback loops and deal with greylisting (because people really love waiting 18 hours for an acknowledgement)

by dingus 7y ago

thaumasiotes 7y ago

I've had gmail send the following to the spam folder: - Legitimate class action notices related to Amazon purchases. - Email coming from addresses to which I had already sent email. (!) - Email from my landlord. - Email coming from Google itself. Based on the contents of my spam folder, which I have to check fairly often because of the extreme overaggressiveness, I would be vastly better off if nothing ever got filtered at all. [1] >> There are going to be false positives, we will make mistakes, but we certainly care a lot about fixing issues like this when we hear about them. This doesn't sound honest, or at least not complete. People have been complaining about this for years. I have personally been complaining about this for years. The loss of obviously legitimate email is completely outrageous. It doesn't look intentional (look at that fourth category!), but it certainly doesn't look like anyone is trying to address the problem. [1] Yes, if spam filtering was disabled, more spam might get sent.

bryanrasmussen 7y ago
>Legitimate class action notices related to Amazon purchases. You'd think Google had an incentive to get that delivered.
mintplant 7y ago
> - Email coming from Google itself. Yep. I missed an invitation to a Google-hosted event at a conference I attended because the email (from an @google.com address, no less) got caught in Gmail's spam filter.
- zwkrt 7y ago
  This is a problem with ML approaches, right? Instead of water boiling at "100C" it boils at "99.98C +- 0.04C". Normally this is ok, but sometimes it isn't!
  - yeahitslikethat 7y ago
    If something happens a trillion times a day with .00001 error rate, then 50 MILLION things went wrong, affecting, say 50,000 people. 50 of them blast twitter and it seems the world is collapsing. The world is resilient.
    - thaumasiotes 7y ago
      I don't think an average of 1,000 mistakes per person-affected-by-mistakes is realistic here. I'd bet it's closer to 1.
      - deleted 7y ago
        [deleted]
    - jodrellblank 7y ago
      If there are 50,000 people and 50,000 people experience problems, that's bad. If there are 5 million people and 50,000 experience problems, that's fine? Isaac Asimov's comments about world population increase involved something about this; the more people there are, the more each individual is dehumanised and rendered irrelevant (my paraphrasing).
      - yeahitslikethat 7y ago
arpa 7y ago
You know what's really funny? That even with that overly agressive spam filter, once in a while (once a quarter maybe), it somehow manages to miss obvious BuY@@NiGERiAn@@Vi@gRa-Cia1is type of emails... which, by the way, my morally outdated spamassasin marks as spam.
- AceJohnny2 7y ago
  "morally outdated" ?
  - SXX 7y ago
    I would make an educated guess that "arpa" is from EX-USSR and their native language is Russian. It's direct translation of idiom "морально устаревший" which literally means something is: * Available for decades. * Far from being top notch technology. * Sometimes of course it's literally mean outdated. Like if you run older CentOS or Debian with decade-old packages. So it's doesn't mean SpamAssassin is bad, but it's very far from state-of-the-art ML technologies that Google might have.
    - blago 7y ago
      Bulgarian has that too: "Морално остарял"
ljm 7y ago
Before I switched to fastmail, gmail would not believe me when I marked an email as not-spam. They would still get flagged. All I wanted was updates from one band. That didn’t stop me from repeatedly getting junk from every other record label my email was sold to. It was an endless procession of shit I never subscribed to.
- StudentStuff 7y ago
  "Inboxing" with Gmail & Outlook is a baroque, hellish process for mailservers. When either provider decides your small email server is sending spam (eg: sending an email with an attachment, or any kind of form email like a daily report) you won't get through to user inboxes, and instead you'll be routed to spam, or for Outlook.com hosted addresses they will accept mail from your server and send it to /dev/null. Gmail's process is bad, but Microsoft has decided to accept emails and throw them away (which is ridiculous).
  - chupa-chups 7y ago
    At my workplace we have two email addresses, one from google mail (gmail for business) and one from MS Exchange. I never had a problem with gmail for business regarding spam. I regularly receive mails from smaller businesses (some of them hosting their own mail server) and never had a complaint from anyone yet. Since i can also be contacted via phone i'd know. On the other hand, MS Exchange constantly delivers obvious spam mails and (quite seldomly, but still) swallows legitimate mail. Anectodal, i know. And disclaimer: the behavior depicted in the article is as bad as it gets, if everything is as described.
  - int_19h 7y ago
    As far as I know, Gmail also does this sort of black hole spam filtering - the stuff that you see in "Spam" isn't everything.
cpeterso 7y ago
Your comment motivated me to check my Gmail spam folder and 10% (3 out of 30) were false positives, two of which were pretty important. :(
- ModernMech 7y ago
  Me too. I found an invitation for an on-site job interview I would have 100% missed if they had not called me as well.
  - influx 7y ago
    Likewise I found they filtered an e-mail from my bank about my car loan.
- wingworks 7y ago
  I've recently started checking quite regularly my spam folder as I've noticed more and more legit emails ending up there. One of them being support emails from TradeMe (one of New Zealand's biggest sites), keep getting put into spam, even after multiple "mark as not spam", along with some other kinda important emails from TradeMe. I've had to put in manual filters to force an email from TM to skip the spam.
  - PenguinCoder 7y ago
    And yet, in my own none Gmail hosted email (fastmail), I currently have 2/55 false positive spam emails. I very rarely ever check it. To note I usually get one actual spam (non newsletter blog spam) to my actual inbox, a month.
  - mr_toad 7y ago
    TradeMe, like eBay and Banks is a popular target of phishing emails.
- baudehlo 7y ago
  I recently missed an email from the UK government regarding a passport application, it went to Spam. Talk about missing the mark!
type0 7y ago
About accountability, this gmail thing vaguely reminds me about "The Drop" situation on the mobile phone network https://www.youtube.com/watch?v=pCOCKS5AJI8 https://www.youtube.com/watch?v=pCOCKS5AJI8
tlogan 7y ago
And all parents with kids in SF public schools should note that all emails from noreply@sfusd.edu (bulletins, etc.) are marked as spam.
preinheimer 7y ago
> - Email coming from Google itself. If email coming from google itself got special treatment there would be masses with pitchforks complaining about that. You've made a lot of good points, but I don't think that's one of them.
- pfranz 7y ago
  I don't think that has anything to do with Google needing special treatment. It shows that the company who made the rules and has every privilege to follow them isn't able to.
  - preinheimer 7y ago
    I think that's a fair interpretation and a good point.
  - jrnichols 7y ago
    Valid point. They've become so large and synonymous with email that many people I encounter are actually unaware that there is other email besides "Gmail." It feels like Google no longer has any incentive to follow the rules, and they feel that they are going to be the ones to make the new rules. The rest of us end up having to implement workarounds.
- thaumasiotes 7y ago
  > You've made a lot of good points, but I don't think that's one of them. I disagree. I think in the absence of that point, it would have been hard to say this: > It doesn't look intentional (look at that fourth category!) But also, I think special treatment for trusted actors is a completely appropriate way to handle email delivery, and I also think it's appropriate for gmail to trust themselves to be sending legitimate mail. Blocking their own email makes them look totally incompetent. They absolutely should whitelist themselves. And they should have a way for you to be whitelisted too, if you want to send email.
  - preinheimer 7y ago
    If they whitelisted the major players in the space, I'd worry that the problem would get _much_ worse for everyone else.
sandworm101 7y ago
>> I would be vastly better off if nothing ever got filtered at all. There are layers of filtering beyond what appears in your spam folder, layers that block obvious spam long before it gets anywhere near your account. If every email ever sent to your address wound up in your spam folder you'd beg for filtering.
- thaumasiotes 7y ago
  I'm prepared to believe this. But it's not a defense of gmail's policies -- this suggests that in fact nothing would be lost if gmail eliminated the spam folder and delivered everything that would have gone there to your inbox instead. So why are they doing this?
- badsectoracula 7y ago
  Before switching to shared hosting from my VPS (one reason being that i didn't want to bother with email maintenance and didn't want to have a separate service just for email), i had my mail with (badly configured) Spamassassin that wouldn't delete mail, just add a "SPAM" prefix in the topic. My mail isn't exactly commonly known, but still is one i have for years and was made public thanks to me releasing an Android app once (after which, spam increased dramatically). Even after running it for years, Spamassassin never marked a legitimate mail as spam, so i'm pretty sure that if i wasn't too lazy to configure it to move it to a spam folder, it'd work fine. Stuff did pass through it (at a ratio of one every four or so) but i was fine with deleting those. What i'm trying to say is that from personal experience, i'd be fine with a spam filter that errs on the side of not marking stuff for spam and me deleting whatever goes through manually. Having to see a bit of spam mail is small cost for losing mail i'm actually interested in.
  - deleted 7y ago
    [deleted]
- vbezhenar 7y ago
  I'm using my e-mail pretty much everywhere (me@vbezhenar.com you have it in clear text, every bot will crawl it now, not for the first time, though). I'm too lazy to setup spamassasin yet, so I'm getting a notification for every spam message I got. The only thing that I'm trying to do is to click "unsubscribe" even from obviously spam mails (may be it'll make more harm than good, not sure). So I'm getting around 20-30 spam e-mails per day. I don't think that it's THAT bad. I'm spending may be a minute every day to delete it. And I'm sure that with absolutely minimal spam filtering I would achieve almost perfect filtering.
mathattack 7y ago
I had a communication from a Google recruiter go to my Gmail spam folder. But it still beats no filtering.
eq_sd_ 7y ago
I've also had email from a Google recruiter go to the spam box...
m463 7y ago
I think the SPAM folder is a separate issue (although important) The article discusses mail not being accepted by google/gmail in the first place.
- thaumasiotes 7y ago
  Fair comment. But that issue is pretty simple: there is no good reason for them to fail to deliver email under any circumstances. (This has happened to me too -- someone tried to email my gmail account and gmail completely refused to deliver it. It was pretty embarrassing.) Messages they think you won't want to receive are what the spam folder is for.
  - m463 7y ago
    "fail to deliver email under any circumstances." The amount of spam that would be delivered if they didn't discriminate AT ALL is enormous. They have to read all this feedback and discriminate better.
  - winkeltripel 7y ago
    > there is no good reason for them to fail to deliver email under any circumstances Yes there is. They don't want to carry traffic from anybody from the major email blacklists. If a mail server is on a real, very transparently-managed blacklist, no large provider should be accepting their smtp traffic.
    - acct1771 7y ago
      Yet all the people here trying to administrate servers from residential and VPS blocks of IPs are telling you they're caught up in this list you're lauding as all-knowing and safe.......
will4274 7y ago
I've also had email from Google recruiters (@google.com email addresses) go to spam. I considered this a high level indicator that Google spam filtering is incompetent, not malicious - if it was malicious, they would at least be able to get their corporate emails through.
ma2rten 7y ago
- Email coming from addresses to which I had already sent email. (!) That is understandable. It is hard to validate if an email is authentic. SMTP has no authentication built in. Gmail can't just blindly accept all emails from addresses that you have already sent an email to. EDIT: Look at this example: https://en.wikipedia.org/wiki/Simple_Mail_Transfer_Protocol#SMTP_transport_example https://en.wikipedia.org/wiki/Simple_Mail_Transfer_Protocol#... Anyone can connect to relay.example.com and pretend to be bob@example.com.
- bdhess 7y ago
  Huh? If it's in the Spam folder, they do accept it.
  - ma2rten 7y ago
    I meant accept as not spam.
- muppetman 7y ago
  Not if you use SPF/DKIM/DMARC they can't, that's the whole point of those various additions. All those "I hacked your email and send you a message from you account" I don't get, because I have a DMARC policy that says if you don't pass SPF/DKIM then you get rejected. So try as the spammer might to connect to my mailserver and pretend to be me, they can't, because my mailserver sees they're not authenticated, and the mailserver they're sending from isn't in my SPF records, isn't signing the message with my DKIM key and therefore it gets rejected at the SMTP level.
- deno 7y ago
  That’s what SPF[1] is for, and optionally DMARC[2]. [1] https://en.wikipedia.org/wiki/Sender_Policy_Framework https://en.wikipedia.org/wiki/Sender_Policy_Framework [2] https://en.wikipedia.org/wiki/DMARC https://en.wikipedia.org/wiki/DMARC
Fogest 7y ago
I just cleaned out a little over 100 emails in my Gmail spam filter yesterday. About 80% of what was in there were emails from YouTube giving me notifications of new videos people have uploaded that I am subscribed to. These emails never used to go to spam, but slowly over time more and more of them would end up in spam. It's at the point now where almost all of them from YouTube go to the spam folder. It doesn't make any sense since they are emails from Google, they are emails I even have a filter applied to so that a label is applied to them. Yes I can adjust the filter and choose "never send to spam" but the messages will still show a warning on them saying "This message was not sent to spam because of a filter you have applied". Sure false positives makes sense, but I don't get how the majority of what is in my spam folder would be emails sent by Google.
- Faark 7y ago
  > It's at the point now where almost all of them from YouTube go to the spam folder. It doesn't make any sense since they are emails from Google It makes a lot of sense... people use the spam button as a lazy man's unsubscribe. Youtube adding the bell button, making mail opt in is probably a response to that. I actually give Google a lot of cred for not simply white-listing its own domains. Though spammers would probably find ways to abuse it and make them look bad anyway.
  - ctack 7y ago
    > people use the spam button as a lazy man's unsubscribe This is the small mail server crux right here. If you’re a small mail server and a few of your emails have been spam binned instead of unsubscribed, it would likely lead to your whole server getting shit canned.
knorker 7y ago
> - Email coming from Google itself. To be fair, sometimes Google does send spam.

tankenmate 7y ago

Normally I wouldn't +1 a message because it doesn't really seem to add much to the message. But in this case I will (even though it might earn me ire from some). If I could highlight the parent in bold I would. I've run mail servers since the mid 90's and the experience above exactly mirrors my experience. And my takeaway is the same, GMail just doesn't seem to care. I understand that GMail can't give away too much info on why a particular email ghosted (opsec and all that) but GMail doesn't need to do a better job of explaining to postmasters what good actors can do to avoid being ghosted, and more importantly have that reflect actual real life experience.

russh 7y ago
Add me to the "I've run mail servers since the mid 90's and the experience above exactly mirrors my experience." list.
- kreetx 7y ago
  I think all of you should start some kind of a movement so that this could be heared better at GMail's end.
  - dhess 7y ago
    That's a great idea. I've run my own mail servers since 1999. I'd happily join a union of independent mail operators with the purpose of lobbying Google, Microsoft, and Yahoo to treat us as first-class citizens.
    - walterbell 7y ago
      Start a mailing list? Or is there an existing one for this topic?
      - kreetx 7y ago
        I think you should have a web page where you list your issues, show how you've done all the normal things to be considered a well-behaved mail domain. And then perhaps report your experiences with iinteracting with gmail support on the matter.

_eht 7y ago

I will chime and validate this experience. You will not fully understand or realize the hostility google has towards open email unless you have working experience with email protocols. This is by design. It is my belief that this is intentional and I would love to be corrected if it’s not.

jolmg 7y ago
Reminds me of this email from Linus Torvalds about a Gmail bug that was affecting git: https://public-inbox.org/git/CAHk-=whP1stFZNAaJiMi5eZ9rj0MRt20Y_yHVczZPH+O01d+sA@mail.gmail.com/ https://public-inbox.org/git/CAHk-=whP1stFZNAaJiMi5eZ9rj0MRt...

lossolo 7y ago

I have similar experience and of course if you send exactly the same message but from gmail account, then it's not a spam for gmail anymore..

deleted 7y ago

[deleted]

xtiansimon 7y ago

Just to be clear, your comments are refering to problems with Gmail's Spam filtering. Isn't that a different issue than what the OP is saying--that his mailserver is being blocked? In other words, emails from his mail server wouldn't even make it to Spam folder.

fgonzag 7y ago
Gmail spam filtering and it's mail server are two sides of the same coin. The mail server itself is blocking believed spam messages as a first layer, then the spam filter acts a second layer that marks messages as spam. I guess they don't want to save messages that they are completely sure are spam, the problem seems to be their false positive rate or simply their approach against small operators.

deleted 7y ago

[deleted]

fpierfed 7y ago

This is totally my experience as well. It got so bad that I gave up operating my email server. Anyway, Gmail is getting some heat in this thread and rightfully so. We should however not forget that Microsoft and Yahoo are just as bad if not worse in this respect.

GFischer 7y ago
I´ve unfortunately searched for some customer-friendly e-mail provider and people want Google. I had set up some commercial accounts on Zoho (my own e-mail is on Zoho) but they asked me to migrate to Google, even though it was more expensive.

andrewprock 7y ago

Just checked my junk folder. 9 out of 10 threads marked as spam were false negatives. That's a 90% failure rate.

tty2300 7y ago

I have everything set up correct and gmail even accepts 90% of my emails but every now and then it randomly marks one of my emails as spam. A gmail user emailed me first and I replied with a text email no links and that gets marked as spam.

emmelaich 7y ago

> There is no tool available to me to understand I _think_ this is because G doesn't want people gaming the system. > or correct the "problem". I regularly fish mail from my spam bin. From mailing lists, and other important stuff -- and indeed Google's own mail! One thing you can do is to get people to add you to their contacts list. It's a hard problem. There is one "solution" you probably do not want. Have Google (and other companies) give their imprimatur to certain mail senders.

slacka 7y ago

First of all, if you check this site: https://toolbox.googleapps.com/apps/checkmx/check?domain=tablix.org https://toolbox.googleapps.com/apps/checkmx/check?domain=tab... It is not to be set up correctly. Now in your case, you failed to mention DMARC records. ALL of the big cloud email services have required that for years. Your SMTP server checklist is straight out of 2005. This entire thread reminds me of the guy who pulled out of a desk drawer an Analog Startac cell phone and blamed AT&T for it not working. Email has moved on. It not Google's fault that people are configuring SMTP servers like it 2005.

mirimir 7y ago
What does this mean? > error SPF must allow Google servers to send mail on behalf of your domain. So you must let Google spoof your domain? That seems crazy. ProtonMail has the same error: https://toolbox.googleapps.com/apps/checkmx/check?domain=protonmail.com&dkim_selector= https://toolbox.googleapps.com/apps/checkmx/check?domain=pro...
- matrixagent 7y ago
  He's wrong about what this tool provides – namely information for people who want to use GSuite for their email. In that context it makes sense that you have to allow Google "spoofing" your domain, of course. But it has nothing to do with what is being discussed here.
  - mirimir 7y ago
    Ah. Thanks. It makes sense now. I didn't read carefully enough.
roca 7y ago
This tool checks that you have set up your domain correctly for use with Google Apps. For domains that are not using Google Apps, its results are meaningless.
zlynx 7y ago
It turns out DMARC isn't actually required. Having it might help your acceptance score, but gmail and outlook don't actually require it. I know because I haven't set up DMARC for my email domain, and my email still gets accepted by everyone. I believe it helps that I accept on TLS with a valid certificate, and I used Google's postmaster site verification.

craz8 7y ago

I sent mail from my personal gmail to my company email (different user) for expenses Since money was involved, I paid attention and followed up Our finance person found many emails in Spam that were important and should not be there On the other hand, my personal Spam folder is certainly full of crap I never want to see. But now I don’t trust the system, so I have to scan them anyway

travbrack 7y ago

Datapoint: Ive run a small exchange server with fewer than 500 users for 5 years and have never had an issue delivering to Gmail except a couple times when we had legit issues on our end.

wildduck 7y ago

Have you looked into WildDuck.email? I find it very nicely made for people who wants to run their own email. Still in Beta. FYI, I'm just a js developer not related to them.

flurdy 7y ago

I also have a 15+ years old set of mail servers and I host mail domains for 10ish friends and family. In fact, I once wrote how to set this up (http://flurdy.com/docs/postfix http://flurdy.com/docs/postfix) I no longer use Gmail myself but half of my users relay some aliases to their main Gmail account. No problems with that, except my servers continuously get rate limited by Google: Our system has detected an unusual rate of 421-4.7.0 unsolicited mail originating from your IP address. To protect our 421-4.7.0 users from spam, mail sent from your IP address has been temporarily 421-4.7.0 rate limited. Please visit 421-4.7.0 https://support.google.com/mail/?p=UnsolicitedRateLimitError to 421 4.7.0 review our Bulk Email Senders Guidelines. I have the normal anti-spam features set up (SPF, DKIM, DMARC, Postgrey, Spamassassin, etc) so most spam gets rejected but not all. But several of my users have very common English names as their aliases that are already guessed and added to many spam databases. This leads to family phoning me all the time to say my server is broken when it is just Google throttling everyone's emails. I frequently have to check Postfix queues and clear some obvious spam or just pointless Facebook update emails which seems to be the majority... (Yes your email admin can read your email...) The only way to improve is to constantly remind my family to not sign up to all crap, use not so common aliases, and try to keep tightening my anti-spam configurations. But we really are not talking about a lot of emails. Probably less than a hundred per day spread across 5-ish end accounts of which 95% is probably legit. Yet Google is treating me as some totally open relay. (╯°□°)╯︵ ┻━┻

flurdy 7y ago
I do understand Google's challenge with handling probably the largest amount of email traffic in the world, and then the most spam in the world, and that it is a continuously moving target. But it does feel like they are treating nearly all minor relays as spam relays. We can not block 100% of spam before we relay onwards to Gmail as that would mean too many false positives emails get blocked, but most try to block as much as possible. In my case, my servers probably block 99% of spam, but some will get relayed, and most of those ends up being handled by Gmail's even better Bayes scoring and filtered to end users' spam folders. I don't know what the automatic threshold limit is to be on their naughty list, but it must be very low, as in double digits per day.

hakfoo 7y ago

A few years ago, I was dealing with a few small business sites which were self-hosted and always had deliverability problems with the large mail services (mostly, back in the day, AOL and Yahoo). Large providers tended to have a world view dividing all senders into two categories: 1) Bulk senders who are clearly mailing the same spam to a list of a billion addresses 2) Non-commercial individual hosts which should be sending five messages a day or less in total. It felt like there was a huge missing third category for transactional emailers nobody wanted to acknowledge. They are probably difficult to score fairly. A hundred "Order details" emails are going to have the same level of randomness/templatedness as the old Viagra spam which had a random block of Project Gutenberg text pasted at the end to trip up filter math. You're not going to have a clear history of "this address bounced twice, let's stop sending newsletters" when most of your messages are to first time customers or once-every-few-years return ones. A lot of the messages will look generic because they use default shopping cart templates. To the extent they provided sender guidance, it was focused around use case 1) -- sign up for feedback loops and deal with greylisting (because people really love waiting 18 hours for an acknowledgement)