35 ms·

Google Is Eating Our Mail

• pettycashstash2 7y ago
  Add another perspective to this. Once I updated my phone, and google would not accept new google Authenticator codes after set up. Locked out of 15 years of email.
  • tyingq 7y ago
    You probably tried this, no idea if it ever works, but in case someone reads your comment and is in the same boat: https://support.google.com/accounts/contact/forum_ar_escalations?hl=en https://support.google.com/accounts/contact/forum_ar_escalat...
  • jryan49 7y ago
    Did the backup codes not work either?
  • lotsofpulp 7y ago
    They specifically provide 10 one use codes for this type of situation that should be saved somewhere. Also, email should be backed up locally if it’s important, just like anything else. I’m not sure what else people want from a secure email service you don’t have to pay for. Also, any work around the 2FA by a human simply means less security for everyone.
    • asdfasgasdgasdg 7y ago
      Right. If there was someone you could call about this, there'd be someone a social engineer could call about it.
  • laurentl 7y ago
    I had a fright last time I changed phones. After restoring my backup onto the new phone, I was about to wipe the old one; I did a final pass through my apps just in case... and saw that Google Authenticator configuration did not carry over to the new phone (which makes sens from a security perspective but is a PITA to manage). I had to re-enroll my device with all the services on which I use MFA. After this episode I made damn sure I had recovery codes stored in a safe place.
    • kkreamer 7y ago
      Use Authy, which does carry over.
    • fencepost 7y ago
      You can also have more than one device set up for TOTP - phone, previous phone or tablet, desktop using WinAuth or similar. Authy and the password managers will also track those seed values, though it's best to store them separately from your actual password storage. Another thing that will not migrate phone to phone is Signal conversations if you're inclined to keep those.
      • laurentl 7y ago
        Yes, I lost my Signal conversation history as well and had to be re-added to all group conversations (another PITA). How do you store seed values in password managers? More specifically: how do you export them from Coogle Authenticator? (I’ve not found that option). And how do you import them again?
        • fencepost 7y ago
          I'm unaware of any options to export, that would be a security issue. To store the seed values, simply store the text provided for use if you can't use the qr code.
          • laurentl 7y ago
            ISTR at least one service (AWS or Google) asking for the first 2 codes after scanning the QR code, probably to sync. So I always assumed simply re-scanning the QR code wouldn’t be enough but maybe it is this simple.
            • fencepost 7y ago
              The underlying technology behind (almost?) all of these is TOTP (https://en.wikipedia.org/wiki/Time-based_One-time_Password_algorithm https://en.wikipedia.org/wiki/Time-based_One-time_Password_a...) which pretty much just depends on both systems having clocks that are reasonably close to synced. The initial value is basically a random number generator seed, and given the seed and a number of iterations (based on the time differential from a set starting point) calculating a code is fairly simple. There would be problems on fully-isolated systems experiencing clock drift, but on any modern Internet-connected system using NTP or on any cell phone with time synced to the network it shouldn't be a factor. The most likely problem scenario is probably a corporate network using only an internal time source that drifts. Doing a single code as validation only makes sense to catch transcription errors since in case of problems someone could end up locked out of an account.
  • dazc 7y ago
    I updated my phone recently and google authenticated directly from my phone number.
  • nf05papsjfVbc 7y ago
    For the future, if you use yubikeys, yubico authenticator is a drop in replacement for google-authenticator and you can save your google-auth setup on multiple yubikeys. So, even if you lose one, you still have it on another. Then it is independent of the phone. Also the yubikeys themselves can be used as 2FA for google accounts. However, sadly, you can't set them up on firefox the last time I tried.
  • deleted 7y ago
    [deleted]
  • ComodoHacker 7y ago
    Happened to me too. The thing that triggered lockout was attempt to login from different country (I've changed my VPN provider). I didn't give Google my phone number so the only option left for recovery was tricky questions like "what city you've been logging in from" which I failed to answer.
• cosmin800 7y ago
  I've always thought that, I also used to operate my own email server and I've encountered the same problem with other email providers, always the big ones, yahoo (there was a time when yahoo was big in email, at least here), microsoft and now yes, google. What is even worse is that sometimes google just drops the email messages, replies with 250 ok, but the email never gets delivered to inbox or even spam. Definetly the big 4 or 5 or whatever are taking over on all aspects of our life. Actually if I think better about it, they have already done so, there was an article here on hn about blocking google, amazon and ms network blocks, nothing really works as expected without them.
• i_dont_know_ 7y ago
  As someone who runs their own personal mail server, this annoys me to no end. I set up DKIM, SPF, and reverse-DNS records and resented every moment of it. Even after all that, there's some chance that an email from my server will be marked as insecure/spam or otherwise just not be delivered because Google has come up with some new brilliant mail security/auth/permission scheme that the world has to adopt tomorrow or be cut off from all Gmail users.
  • macspoofing 7y ago
    >Even after all that, there's some chance that an email from my server will be marked as insecure/spam Don't I know it. This happened to us but not with Google. We use Exchange Online (Office365) and have DKIM, SPF headers configured. At some point our emails stopped being received by bell and cox emails addresses ... sporadically but sometimes for days at a time. It turned out that the spam filter provider (used by bell and maybe cox) correlated one innocuous phrase in our confidentiality notice (which is appended to all outgoing email signatures) with spam and therefore marked the entire email as spam as well.
    • Leace 7y ago
      To be honest I don't like this confidentiality notes and from what I've read (not a lawyer) they don't have any practical effect either.
      • macspoofing 7y ago
        It's there, it wasn't my call to put it in.
      • reaperducer 7y ago
        I always giggle when I see those. Especially the part about how "if you are not the intended recipient, you are ordered to destroy all copies of this message." It's my computer. You sent the bad message. You're in no position to "order" me to do anything. I suspect they're mandated by self-important Lower Middle Manager of the Year Award nominees, not actual legal departments. The sort of people who never do anything but push papers around, yet put a bunch of meaningless initials after their name in e-mail signatures because they went to a conference.
• superpie 7y ago
  A while back I'd explored trying to own my data, especially with email, and found that the efforts involved in hosting your own email server were tantamount to a full-time job. The amount of fighting you have to do to stay on everyone's whitelists is absurd.
  • ocdtrekkie 7y ago
    There's a good middlespace here, where you pay someone for email service who isn't an adtech company. Sure, you don't host it yourself, but someone who's not selling you is and you're their actual customer. (I pay FastMail, personally.)
    • Nasrudith 7y ago
      I think you are still being sold even if you pay. Nothing really stops that. Even if they give away or even pay the bigger providers they are ultimately selling a white list. Since if they just accepted whoever gave them money with zero policing of abuse, even a warning and take down system for spam then nobody would accept their whitelist as it eventually becomes a blacklist of "the one spammers use" and their business model essentially becomes a scam.
  • raverbashing 7y ago
    That's the cost of not pushing the cost on those responsible: the spammers
  • megous 7y ago
    There's some setup cost, for sure. But there need not be that much of an ongoing maintenance (for me, it's just pacman -Syu + restart of some services, perhaps 10mins a month). You can also gain some interesting features and insight as a result.
• reaperducer 7y ago
  Google has no incentive to fix these kinds of problems. It's big enough that when someone complains that a message sent wasn't received, the intended recipient will say, "I never have problems with my Gmail account. It must be you." And the sender has to switch to Gmail to reliably communicate with the outside world. I wish this was just paranoia, but we've seen multiple discussions on HN about Google programs and policies that alter the internet in ways that only benefit Big G. It's like we're heading back to the days when people didn't know the difference between AOL and "the internet."
  • nothis 7y ago
    The evil part about this is that it needs no one at any step to say it out loud. It just “happens” as a result of Google being big. Whatever lead to Microsoft being sued for pushing Internet Explorer in Europe and whatnot? Shouldn’t that kick in already for at least some of the recent Google bullshit?
    • Nerdfest 7y ago
      I think it should, or some sort of oversight should be in place. Their filters are extremely aggressive. I have a specific GMail rule in place to ensure that all mail gets through and nothing gets sent to spam, but even with that the occasional message gets sent to spam. It's generally advertising, but never actually spam. There's definitely for GMail to become the new "Windows", with everyone else fighting to stay connected.
  • dralley 7y ago
    They don't even have to be doing it deliberately; as you say, the lack of incentives to fix such issues will cause a gradual evolution in that direction.
    • cptskippy 7y ago
      I feel like as they leave more and more up to the algorithms, we suffer. I've been having this ever increasing in frequency issue where Gmail Search just doesn't working. Not that it doesn't return search results but rather that it returns what I know to be incorrect results. I have an email with some account information that I access a couple times a year at most but will consistently need at least once a year. Just infrequently enough that I can't remember the exact contents. I know exactly what the subject line of the email is and it's a suggested search query in Gmail but about 2 years ago Gmail just stopped being able to find it. I can navigate to the email via label or star just fine but I can no longer search for it. Is it because it's an 8 year old email? idk what the reasoning is but Gmail as deemed it irrelevant and stopped returning it as a search result. This isn't a platform issue either, it happens in App, or browser on multiple devices.
• maxaf 7y ago
  > I'm sure there is no malicious intent behind this How can anyone be sure of this? This is only one of Google’s practices that seems to follow a pervasive pattern of eroding open internet standards while presenting Google’s own proprietary implementation as somehow superior. Eventually, the open standard loses all meaning because the most popular implementation does not actually adhere to it. Meanwhile, Google reaps enormous benefits in the form of additional signals for its advertising business. How can this not be grounded in malicious intent? > and that there are some very smart people working on spam prevention at Google. There are some very smart people working on advertising at Google. The rush to forget the primary nature of Google - it’s an adtech firm - is why they’ve been allowed to skate for so long. Gmail’s spam filtering is just a pretext for passing all email through a machine learning system. Sure, one possible signal emitted by that system is whether a message is spam or not. Perhaps this determination is conflated with wether the message is useful for ad targeting: after all, when viewed from Google’s own perspective certain e-mail messages contain no information which can be used for ad targeting, so they must be spam. The user’s interests are clearly secondary to this. So, back to the “smart people” working on this: at what point do we begin judging engineers for working at Google? There’s a lot of highly vitriolic criticism that emanates from Google’s workforce on a variety of subjects, but how many of them would actually pull the pin and leave their employer? I don’t have any statistics to offer, but it seems to me that we still have a ways to go before Google has become completely drained of engineering mindshare.
  • upofadown 7y ago
    Indifference that causes a business problem turns into not-indifference. Indifference that does not cause a business problem remains indifference. No one ever calls a meeting. There is a lot of evil done in the world that comes from this simple and straightforward principle. You don't have to have an evil plan to end up doing evil. Personally I see this as just more Google incompetence. Gmail has just become bad. If you want to get all your mail you have to use something else.
    • reaperducer 7y ago
      Personally I see this as just more Google incompetence. Gmail has just become bad. If you want to get all your mail you have to use something else. It's like we're going back to the days when each e-mail service only trusted messages from its own users, and if you wanted messages to go to another network, you had to use an e-mail gateway. Can I get my ...!killer!jolnet bang path back?
• sneak 7y ago
  I have run a private mailserver for over 20 years, and I have the same problem. There's no telling what Google will let through. I have a second account at a medium-sized email provider, basically paying them for access to the email deliverability cartel.
  • rc_mob 7y ago
    We need a new message protocol
    • Funes- 7y ago
      So Google--or any other big corporation--can take it over as well? I don't think coming up with yet another open protocol is the solution.
    • tqkxzugoaupvwqr 7y ago
      I fear this will lead to the same problem: If it’s interesting enough for Google, they will offer a free service, gobble up a large number of users, at which point they can hold the new protocol hostage, add their own extensions, and everyone else must follow.
      • sneak 7y ago
        It’s called XMPP, and Google ran a public gateway federating for Google Talk. Basically nobody used it, and what little use it did see was mostly third parties sending spam to Google users over the gateway, so they shut it down. Centralized systems seem to give users what they want. This is why iMessage, FB Messenger, WhatsApp, Snapchat, and Instagram are huge, and email is dying (and XMPP failed to get any real traction in the first place).
    • Silhouette 7y ago
      There's nothing wrong with the basic principle of email (other than its lack of built-in security, arguably). The problem is that certain large organisations now claim to offer an email service, but don't actually make it work properly. It might have been better for the rest of the community to block those services until they did work properly to force everyone to play by the same rules, but unfortunately we have reached a point where some of them are too big for that to be a practical solution. The obsession with fighting spam and malware means some mail services are now far too ready to accept false positives where they block legitimate mail, in order to reduce the risk of false negatives where they let illegitimate messages through. This seems strange, because often the false positive will be more damaging. In the cases where it is not, such as messages being sent with known malware attached, the receiving mail service could still deliver a short replacement message to the intended recipient instead, notifying them that something was blocked because of malware and providing essential details like the sender and subject line, so the recipient could take further action rather than miss something important.
• stratosmacker 7y ago
  I have a similar problem and it sounds like some here do as well. What can we do about Google's email monopoly?
• surgi 7y ago
  Am I the only one here experiencing that GMail does reasonably good job in filtering out the spam, while I have to visit the Spam folder approximately once a week, just to randomly correct a mail or 2 a month? (not a Google employee)
  • techsupporter 7y ago
    That’s not the issue, though. The issue is that Gmail are too zealous in blocking and wind up blocking legitimate email with no way for the sender to debug what happened. From your perspective, nothing happened because how do you prove you never got an email you never knew was sent to you? The people having problems here are non-Gmail senders sending email to Gmail recipients.
  • andai 7y ago
    Every time I look in my spam folder, there's dozens of legitimate emails in there, and I get this dreadful feeling that over the years, thousands of legitimate mails went over the 30-days-in-spam threshold, down the memory hole forever.
• jasode 7y ago
  >I've been running a small SMTP and IMAP mail server for [...] around 15 years [...] I have SPF records and DKIM message signing setup on the domains I use. The server is hosted on commercial static IP space (with the very same IP it first went on-line) and I've made sure with the ISP that correct reverse DNS records are in place. [...] Being a good administrator and a well-behaved player on the network is no longer enough [...] Now every time I write a mail I wonder whether Google's AI will let it through or not. [...] So far Google has blocked personal messages to friends and family in multiple languages, as well as business mail. I stopped guessing what text their algorithms deem suspicious. Yep, I know the author's frustration very well. I made a previous comment[0] trying to warn others of personal email servers' outgoing email being spam-holed -- and yet some of the replies still argued I was overstating the difficulties. Everybody's risk tolerance is different. Personally, I just don't have the bandwidth to administer my own private email server and constantly worry if recipients are receiving my emails. [0] https://news.ycombinator.com/item?id=15526089 https://news.ycombinator.com/item?id=15526089
• jdmoreira 7y ago
  I had exactly the same kind of problems when I tried to set up a new mail server. In the end I just gave up and started using mailgun as a relay.
  • silversconfused 7y ago
    Can you tell us a little more about your setup? I have a home email server, incoming only because of ISP restrictions, and I would love to make it into a real email server for my family and business.
    • jdmoreira 7y ago
      It's just smtp relay really, nothing fancy. I still run my own mail server but I relay outbound over to mailgun
• reaperducer 7y ago
  The last time a company screwed with e-mail at scale was Earthlink. Remember for a couple of years, every time you'd send a message to someone at Earthlink, you'd get an automated rely demanding that you verify yourself before the message would go through? Now I can't remember the last time I saw an @earthlink.* e-mail address. Google has apparently learned from that and put the "error" in the 550 messages, where they can't be seen by the end user, and lead to non-helpful resolutions for sysadmins. The result is that the blame for missing messages goes to the sender, not to the recipient's email service. If Gmail at least notified the sender that there was a problem, then a pattern of responsibility could be established. But this is just another dark pattern.
  • AJ007 7y ago
    If enough expected messages don’t show up, users are going to start thinking gmail is broken. The problem for Google is that they are not the only trusted brand consumers interact with. I’ve seen a lot of stuff start ending up in spam that shouldn’t be there. If enough password resets, customer service responses, and order confirmations aren’t delivered, it looks like Google has a problem. If it doesn’t even make it to the spam box it makes Google look really bad. Google has definitely been changing things, everyone presumes it’s some rogue ML system. My company had solid Gmail delivery for almost 10 years, and at a fairly large volume. Now we can’t get it delivered any more. Very close to showing users a message not to use Gmail.
    • luckylion 7y ago
      > If enough expected messages don’t show up, users are going to start thinking gmail is broken. I doubt that. They will rather consider that the people (or their IT) that send them emails made a mistake. It's Google after all.
      • bluGill 7y ago
        If several different IT departments all start sending that message though, people start to think maybe google has lost it...
• deleted 7y ago
  [deleted]
• wichert 7y ago
  There may be a workaround here: configure your own mailserver to relay outgoing messages through a service with a known good reputation. For example relaying via sparkpost or mandrill should be trivial to do. You might be able to do that under a free developer account if you don't send a lot of email.
• tambourine_man 7y ago
  Same here. I love email for a variety of reasons, one of them for being an open standard and protocol. So, naturally, as many of you, I went the mail/postfix, DKIM, SPF, etc way. And all is fine until you start receiving random hard bounces with no real debugable answer for Google. It got me deeply sad and questioning my decisions: since you can’t really ignore Gmail, email isn’t in practice “open” anymore. So I might as well sign up for Facebook, WhatsApp and the likes. It’s been years and I haven’t yet, but it’s getting harder and harder.
• torqueTorrent 7y ago
  Companies like this give whores a bad name.
  • sctb 7y ago
    Please stop posting unsubstantively. https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
    • torqueTorrent 7y ago
      I'll do you one better! In the spirit of the pre-commercialized, pre-monopolized and pre-whored-out Internet, and the extent to which freedom dies with a wimper on Web 2.0, I'll stop posting on HN entirely!
• petercooper 7y ago
  Lest anyone think Gmail is less susceptible to problems because of the sheer volume of mail they handle (so any problem gets a lot of 'eyes' on it), they had an amusing and long-standing bug that meant any email containing a link to any domain name starting with "0x" automatically went to spam. You could literally have two long standing, legitimately used accounts send an email to each other containing a link to a URL like http://0xANYTHINGHERE.com/ http://0xANYTHINGHERE.com/ and it would be insta-spammed. I suspect it was a hard coded rule to avoid people using "long IP" URLs to circumvent other filters.. except there are lots of legitimate 0x domains that aren't long IPs. It was fixed sometime in the past year but I got a lot of use out of it in talks I've given about email deliverability over the years.
  • lupire 7y ago
    What is your favorite public 0x domain?
    • petercooper 7y ago
      https://0x46.net/ https://0x46.net/
• deleted 7y ago
  [deleted]
• beams_of_light 7y ago
  This is the reason I sadly don't recommend to customers that they run their own email servers, and instead outsource it to GSuite or O365. Being on the hook when emails aren't delivered for one or more of a dozen reasons that you can't control is no fun at all.
• rootusrootus 7y ago
  Reminds me of Spamhaus. Are they still around? When I worked for an ISP we had to deal with them regularly. They'd happily hold you hostage over a single e-mail.
  • jrnichols 7y ago
    yes, they are, and they've been really really good. haven't seen any of the behavior you refer to. ever. now SORBS on the other hand... doesn't really exist anymore (fortunately) but spamhaus has been great. especially when used as a filter in Postscreen.
• bartimus 7y ago
  Google seems to not agree DKIM is setup properly? https://toolbox.googleapps.com/apps/checkmx/check?domain=tablix.org https://toolbox.googleapps.com/apps/checkmx/check?domain=tab...
  • proaralyst 7y ago
    There's no standard DKIM selector though, right? You may have used a different DKIM selector in your query to the one they have set up
    • LeonM 7y ago
      Google probably stores the selectors for a domain when it processes email from that domain.
  • supermatt 7y ago
    Not set up to send email through google. Unless they are using gmail as an smtp relay (which they aren't), that shouldn't be a problem - otherwise he would be giving google authorisation to send emails on his behalf.
  • avian 7y ago
    Thanks for the link. I think the tool you linked specifically checks whether DKIM is set to allow Google's servers to send mail on the behalf of your domain. These checkers for example don't find any errors with the DNS record: https://dmarcian.com/dkim-inspector/?domain=tablix.org&selector=alpha https://dmarcian.com/dkim-inspector/?domain=tablix.org&selec... https://mxtoolbox.com/SuperTool.aspx?action=dkim%3atablix.org%3aalpha&run=toolpage https://mxtoolbox.com/SuperTool.aspx?action=dkim%3atablix.or... One thing that this second one does find is the lack of a version number in the record. I see that the RFC [1] got updated since the last time I checked and having that is now recommended instead of optional. I'll add that. Thanks! EDIT: after adding version record, Google's tool now shows green checkmark for DKIM as well. In any case, it's impossible to test a DKIM setup just by looking at the DNS records. The true test is to verify the actual signature in the message. Last time I checked, the mail on the receiving end (Gmail), did have "dkim=pass" in the Authentication-Results headers. [1]: https://tools.ietf.org/html/rfc6376#section-3.6.1 https://tools.ietf.org/html/rfc6376#section-3.6.1
• jbergstroem 7y ago
  I'm also one of those oldfarts that insists on hosting my private emails for me and a few friends. I've owned the ip since early 2000's and have had zero "spam"/blacklist incidents. One trend I've been noticing is that Microsoft/Outlook have regularly started to block me. I have to contact their abuse which takes a day or so to get unblocked. To date, I've done so >10 times: hotmail-com.olc.protection.outlook.com[104.47.1.33] said: 550 5.7.1 Unfortunately, messages from [<redacted>] weren't sent. Please contact your Internet service provider since part of their network is on our block list (S3150). You can also refer your provider to http://mail.live.com/mail/troubleshooting.aspx#errors. [VE1EUR01FT028.eop-EUR01.prod.protection.outlook.com] (in reply to MAIL FROM command) Reporting-MTA: dns; <redacted> X-Postfix-Queue-ID: 14A41FEB66 X-Postfix-Sender: rfc822; <redacted> Arrival-Date: Thu, 14 Mar 2019 14:07:42 +0200 (CEST)
  • _dark_matter_ 7y ago
    At >10 times I'd automate that process.
    • sophacles 7y ago
      Hrm, an automated appeal to automated spam-listing probably looks a lot like spam to the algorithms used to detect spammers.
    • jbergstroem 7y ago
      Form has captcha: https://support.microsoft.com/en-us/supportrequestform/8ad563e3-288e-2a61-8122-3ba03d6b8d75 https://support.microsoft.com/en-us/supportrequestform/8ad56...
      • ApolloFortyNine 7y ago
        Is it illegal to pay for one of the captcha solvers? Ones like this are the easiest, captcha solves cost something like 4 cents each, and all you do is send the image to an API, which responds with the solved captcha.
      • 7y ago
• LeonM 7y ago
  I run a mail monitoring service [0], and we hear this complaint every now and then. A couple of things that regularly seems to trigger false positives in spam algorithms: - no or misconfigured SPF and/or DKIM - no or misconfigured reverse-DNS - automatically included footer texts (confidentiality, copyright, safe a tree don't print, etc) - regular automatic replies from the domain (such as out of office notifications) - the use of embedded images (logos, human signatures, etc) We sometimes joke that these triggers were built in by the algorithm developers as a means of punishing those who litter their email with pointless texts and images. [0] https://www.mailhardener.com https://www.mailhardener.com
  • gambler 7y ago
    And with all this corporate shills chant that "Hosting your own things is easier than ever before".
    • LeonM 7y ago
      I know it's going to be an unpopular opinion in this thread, but my opinion is that email is one of the things that most businesses shouldn't run themselves.
      • feanaro 7y ago
        Why?
        • LeonM 7y ago
          Multiple reasons, but here are a few of them: - Email is hard to get right. There are many subtle configuration mistakes you can make, and it's pretty hard to detect those mistakes (hence one of the reasons why I started Mailhardener) - Your users will have high expectations. For email to be used effectively users expect email to work on many devices, have a webmail client, calendar integration, a good search function, good spam detection, infinite backups, ability to have huge attachments, etc, etc. - Your spam filter is never going to be as effective as that of the big email services. They have ML based spam filters that are constantly evolving. Even if you were to have access to their ML models, you wouldn't have enough data to effectively train that model. - Hosted email solutions are almost always more cost effective, regardless of the size of organisations. Running and maintaining an email service is labour intensive. Of course, there are also many legit use cases where you want to run your own email server. But for >99% of organisations, it just won't make any sense to run their own email.
• 5874-4b22-a4e0 7y ago
  Op should sign up for gsuite.
• duxup 7y ago
  >I wonder. Google as a company is famously focused on machine learning through automated analytics and bare minimum of human contact It should be noted that ultimately these efforts to "learn" result in ML, AI whatever pointed AT you, not working for you.
  • Nasrudith 7y ago
    Well of course - don't trust the client is a rule for a reason. I get the spirit and how it can suck but it seems to be for a reason. Anything which blindly works for "you" can be abused by other "you"s like spammers and other bad actors.
    • duxup 7y ago
      I think the catch with AI and ML there is "reason" but it's just math or something that happens because. If it is "reasonable" (trying to find a definition here) can only be determined by humans.... provided they care or understand it. I always quote this one but I really like it: “Once men turned their thinking over to machines in the hope that this would set them free. But that only permitted other men with machines to enslave them.” ― Frank Herbert, Dune I would argue that it is clear that to some extent, Google does not care about the end results.
• zoom6628 7y ago
  Only 3 people I know using Gmail any longer. I stopped back in 2010 and "didn't feel a thing".
  • mrweasel 7y ago
    How do you really know? G-Suite has become extremely popular with business in the last few years.
    • megous 7y ago
      You'll see something like this in the logs: ... status=sent (250 2.0.0 OK 1556280611 c1###########x.57 - gsmtp) even if the e-mail address doesn't end with @gmail.com
• Tharkun 7y ago
  I've had Google blackhole me several times. Each time without any apparent reason. It's impossible to talk to a human being at Google who is able or willing to help. I've had similar problems with MS, but at least I was able to talk to a human being and get the situation resolved.
• hazeii 7y ago
  Recently Google has bounced replies to emails from a gmail.com address to us, with a message about it being "suspected unsolicted email" (for a reply???). That's just egregious (we're on a dedicated server at a reputable supplier, same IP for years, proper email processess, not on any blacklist and only low-volume usage - certainly no spam).
  • detritus 7y ago
    Yup, I literally lost jobs because of responses to enquiries hitting GMail's spam trap, before finally giving up and moving from self-hosting to Fastmail.
• alyandon 7y ago
  I have logwatch logs that I email to my gmail address via Google's own SMTP servers using my own Google credentials for authentication. Hilariously, Google will flag these emails sent to myself using my own credentials and their infrastructure as spam. I have no faith in them ever getting this right.
• rayiner 7y ago
  As far as I can tell, Gmail isn’t actually “real email” anymore, and you shouldn’t use it for anything important. It’s verging in being a walled garden.
• lucb1e 7y ago
  Can confirm, my server is also not accepted by google for years now when mailing someone with a business account. I'm not going to bother fixing Google's problems and support the behaviour, though, I'll just contact them through other means and let them know their email provider is blocking mail addressed to them. At work we use sendgrid because of this. Have to trust a centralised third party to send out api keys. It's frustrating.
  • jasode 7y ago
    >Can confirm, my server is also not accepted by google for years now when mailing someone with a business account. Hey, I remember you were one of the replies[1] in my previous thread that said I was exaggerating the "send emails" issue. Maybe your difficulty with Gmail's mystery filtering algorithm will warn others that depending on personal email servers for reliable outgoing email is a non-trivial endeavor. Perhaps personal email servers are not impossible to set up but they're also not as easy to debug as some make it out to be. [1] https://news.ycombinator.com/item?id=15526127 https://news.ycombinator.com/item?id=15526127
    • lucb1e 7y ago
      Good memory! A few things: - My comment in this thread was mainly about gmail for businesses, hence the contrast between "I'm not going to fix their problems and just tell the recipient their setup is broken" and the previous "it's doable". Other mail providers are usually no problem at all, and personal Google accounts are also better. - For email to personal gmail accounts it seemed to be enough to send a few mails and get a good reputation. Since then (and since my previous comment), I think I've started ending up in spamboxes again, but the number of messages I send to Google is very low. It's not outright rejected, though, since I would notice that. - I still think this part of my previous message is very true: "I don't think we should dissuade people from doing it, especially if the fact that more people doing it means that it'll be easier next time because it'll be slightly more common. Many of us are in tech and the field is a small subset of the population. Even if it's a small amount of servers setup by us, that could make a noticeable on those working at bigcorps who write the hostile receivers." > Maybe your difficulty with Gmail's mystery filtering algorithm will warn others that depending on personal email servers for reliable outgoing email is a non-trivial endeavor. I hope not. For what it's worth, I'm still at it and have no plans to stop hosting my own mail! I'm very happy not to share my data with a third party (the ideological part of it), but on a practical level, I also don't have a spam problem and I can be sure that email arrives. No message has ever not reached me due to spam filtering. It's also much faster than, say, Runbox, which we use at work and takes at least 30 seconds for any sign up email to arrive (too short to get coffee, too long not to be slightly annoyed). As another data point, my girlfriend switched from 1und1 to my mail server because it has some conveniences, without most of the downsides (since I manage it all for her). She hasn't had delivery issues so far. As a related tidbit, I've also recently started blocking Google from my website. They're being a dick on the internet (not to me, it's more of a solidarity move, see https://lucb1e.com/!130 https://lucb1e.com/!130) and like with email, I'm not dependent on Google so I am again in the rare position to say "no, I'm not going to play your game" to Google. I'd rather encourage others to do the same than to give up and treat both email and search results as a walled garden.
• aloukissas 7y ago
  Yet one more example of this. See [1], [2]. Definitely Google has done something to mess up their spam filter algorithms in the last year. [1] https://news.ycombinator.com/item?id=19536465 https://news.ycombinator.com/item?id=19536465 [2] https://news.ycombinator.com/item?id=19500357 https://news.ycombinator.com/item?id=19500357
• jussij 7y ago
  I run a highly customised PHPBB forum. Without these customisations that forum would be overrun with all sorts of spam. However, these customisations only stop spam postings but can't stop actual registrations. Based on the users that I see who are registering I see a great majority of these spammers love using Gmail accounts. So while it is good that Google Gmail is trying to fix these spam issues, from where I stand Gmail users seem to be a big part of the spamming problem. Spammers love Gmail only because they can easily create spamming e-mail accounts.
• username223 7y ago
  I'm glad I'm not the only one seeing these problems. I send mail from a personal server, and recently discovered that my messages to people @gmail were frequently sent to spam or simply dropped. I briefly thought friends were ghosting me until I figured it out. I set up SPF, DKIM, and DMARC, but it seems like the only way to get reliable email these days is to use Gmail, Hotmail, or Yahoo (at least for a little while...).
• chrisfinazzo 7y ago
  Can someone explain in simple terms why, in 2019, anyone is running their own mail server? Unless you are the sysadmin, server guy/gal, whatever, this just sounds like signing up for a world of hurt doing it yourself.
  • jamesgeck0 7y ago
    If you've been doing it for 15 years, why should you have to stop?
  • simmons 7y ago
    Contrary to popular opinion, I don't think the technical aspects of operating a mail server are all that hard. Perhaps I'm underestimating the learning curve since I've been doing it for 27 years. (If you're running your own mail server, you are the sysadmin.) The frustration comes with scenarios such as the one outlined in this blog post, where small mail server operators get bullied even though they are doing everything right. I can completely understand not wanting to operate a mail server due to this situation, or not having interest in leveling-up server administration skills. For those of us who do have expertise in running mail servers, it's a shame we have to deal with these obstacles.
    • calvinmorrison 7y ago
      The biggest obstacle of course is dealing with deliverability problems. As most posters have commented here, getting people to accept your mail, and accepting mail from reputable sources isn't an easy problem to solve. At FastMail, a considerable amount of time is spent ensuring customers don't need to worry about deliverability problems or spammy inboxes. To me, that's the biggest reason not to spend time running your own infrastructure for mail. Getting people to accept your mail reliably, is much harder than adjusting some postfix configuration.
  • perlgod 7y ago
    Because email is one of the few edifices left of the truly open internet, and some of us aren’t ready to concede defeat of what was once a free platform to a corporate oligarchy.
• crank190426 7y ago
  In my opinion the Gmail web interface should expose a way to generate something that is similar to a password, whose delivery is then controlled via the web portal. This is similar to the +whatevertag trick that gmail pioneered for tagging, however that can be removed by malicious parties (spammers) via a simple regex. So Google have almost all of the infrastructure but should just add a bit more to get the rest of the way there. What I mean in specific : 1. I want to sign up to and receive your newsletter (you[re Ted) but I don't trust you yet. so I should navigate to gmail.com, click something like "generate another inbox", leave it set it to "For now deliver this mail to my inbox", add the description "for Ted's Possibly Spammy Newsletter", and then click "generate". It should give me inbox3943578423@gmail.com - similar to a phone number but a bit longer and personalized to one recipient - and then I should give that to the recipient to use, in this case the possibly spammy newsletter. It should always be delivered to my inbox, as I've set. Once one of the spammers sells my email address (for example I start getting advance payment scams) I'll be able to disable further spam from there by sending it to the trash but also know that Ted's newsletter is the one that got compromised or sold it. You can do this today by going through the steps of registering a new gmail address and turning on forwarding, but it takes like 10 minutes to do so. it should be like 10 seconds. This should be possible because people always have easy access to the gmail web interface. There's no reason it can't be a bit more like a social network where you confirm it from the web interface as well. that's my idea anyway.
• ddebernardy 7y ago
  What boggles the mind is that they've a very string disincentive to fix it: I had to move away from my privately run email to Gmail for that very reason, and I'd gather I'm far from being alone.
• syntheticnature 7y ago
  Google? Any large organization. I help a small non-profit run their site in my free time, and Google's actually been the nicest about our oldschool forwarders and Mailman lists. Despite being on no public blacklists, AT&T domains drop us at the border, with the only appeal process being via email, and I've never heard back. AOL and Yahoo? Flip a coin. Of course, email forwarding turns out to suck, but we're just going to suck it up and move to G Suite for organizational email addresses and let folks forward from there. E-lists, OTOH, I haven't found a good integration to automate membership in the organization vs. G-Suite; perhaps it's time to just move to a forum.
  • judge2020 7y ago
    FYI Google offers free GSuite basic for nonprofits: https://www.google.com/nonprofits/offerings/apps-for-nonprofits.html https://www.google.com/nonprofits/offerings/apps-for-nonprof...
    • syntheticnature 7y ago
      Indeed, the free GSuite is the plan.
• bigend 7y ago
  Google has become pure evil. Avoid Android. Don’t use Chrome, G-mail or Google maps, or their search. Block Google analytics and alikes.
• DoctorOetker 7y ago
  I wonder if originally this technically constitutes an illegal form of interception of mail in some jurisdictions. Perhaps there really would be a lot more spam without such filtering, but it points to the actual problem being elsewhere. Perhaps we need some kind of cheap and userriendly (uniform but decentralized) email court system, and fine / ban email accounts that misbehave?
• ackbar03 7y ago
  I for one welcome our new overlords
• ameliozanchi 7y ago
  Does AI compares whom data to define which message is important or No? It doesn't track any specific way on evolution of mentality when it goes to define data correlated to humans usage. Like others products on market. Therefore, if you apply this thought to AI I guess they just made with basics conglomerated informations gathering. That CAN´T move a barrier when satisfying humans usage. It is just how you confront which message is correct to say is relevant or no. AI can be freely a way companies uses their needs on market influence because they're , on a very simple association, doing by themselves, then people get a notion of arbitrariety on those machines that create discussion to the whole universe you see about robot revolution etc, but not the case. I guess they try to hard to make then appear as human to be soon accepted by costumers, its just the way it is, it is not the time to require maximum levels of perfection on how easy they're to addapt as a user
• majestik 7y ago
  He’s been using the same IP address for his mail server for 15 years. It’s possible his IP is on a subnet range that regularly gets blacklisted by Gmail due to other actors on the subnet sending malicious email/spam to Gmail.
  • megous 7y ago
    Presumably, Google's smart enough to identify individual IPs in the range that don't send spam. I have a real shitty neigborhood IP wise, having my mail server on a $1 VPS. Public Outlook.com drops all my mail regardless of content for example, citing spammy IP range. Yet gmail somehow learned after a while to pass most of my messages through. The real problem is lack of provided reasons for blocking, so people waste a lot of time trying to figure it out by guessing and trying random shit.
• pera 7y ago
  If you run your own email server check out Postmark's DMARC monitor, it's a free service: http://dmarc.postmarkapp.com/ http://dmarc.postmarkapp.com/ (tablix.org doesn't have a DMARC record)
• deleted 7y ago
  [deleted]
• js2 7y ago
  I had my personal domain hosted with Google years ago. One of the reasons I moved away is that emails that I was sending to other Google Apps domains were bouncing as spam when the destination address was an alias. The bounce came from Google Groups, which is what Google used for handling aliases within Apps domains. e.g. https://pastebin.com/u48DAaLP https://pastebin.com/u48DAaLP That particular example was sent via SMTP, but I had the same problem when sending via the Gmail web interface, and it occurred sending to at least three different Google Apps domains. After I moved my domain off Google Apps (I switched to Fastmail for a variety of reasons, but that issue was the kicker), I was able to send to those same addresses without issue. In fairness to Google, I was on the Google Apps free tier at the time, so there was nowhere to go for support.
• megous 7y ago
  The real issue here is that if you tell some gmail user to contact their e-mail provider to resolve delivery issues on their side, like this, they'll just stare at you, not knowing what you're even talking about. It's like there's an assumption that gmail is perfect, and the problem is with the sender. Even if that was true, a normal mail hosting company would at least tell its customer why the mail is not being delivered, so that the customer can tell the sender what to fix. The gmail recipient is never exposed to this side of google. So they don't know what a nightmare comapny it is to communicate with. Why should everyone and their dog be solving gmail users's problems with receiving messages? It's such a demented system. It should be the other way round. Recipients, via their provider should be solving their issues with spam filtering and blocking. If the gmail user would be blocked by my mail server, I would not tell them to go guess what's wrong, fix gmail, and to have fun. It should not be acceptable the other way round either.
• afarah 7y ago
  Ignore it and let people be angry (at Google for not receiving their mail from various places). Meanwhile move to a decent service like ProtonMail.
• mtw 7y ago
  In the article: > I can't tell other people to go off Gmail I disagree. There are reasons to switch off gmail. Not just Google eating mail but also for privacy reasons. Google knows all about your banking, eCommerce orders, your media subscriptions, health issues and many other dependencies. A good alternative is protonmail. It is private, has a mobile app, is a free but you can also pay to support the service. I also consider protonmail much more secure than gmail.
  • craftyguy 7y ago
    Or, if you don't want to commit yourself to yet another walled garden, there are quite a few other email providers that to choose from.
    • silversconfused 7y ago
      Please list a few. I need 3 personal inboxes, 2 home business inboxes (bills, etc), and the ability to make new inboxes for stupid startup prototypes you buy a domain for, hack on for a few days with dreams of huge sales, and then forget about for a year until the domain expires. You know, normal hacker stuff.
      • gsreenivas 7y ago
        You may want to check out Helm - https://thehelm.com https://thehelm.com (I'm a co-founder)
      • jrnichols 7y ago
        https://mxroute.com/ https://mxroute.com/ maybe?
      • craftyguy 7y ago
        Here's a few: https://www.privacytools.io/providers/email/ https://www.privacytools.io/providers/email/ Personally, I currently use mailbox.org.
• mises 7y ago
  I don't know that this is purposeful in all cases. I've had issues on a mail host where sending an email to a different box on the same domain would reject the message as spam, refusing to even deliver it. With respect to google, however, the fact that they offer a service of their own that conveniently can avoid this issue is more than a little shady.
• vectorEQ 7y ago
  They want your users to use gmail, that's all.
• c3534l 7y ago
  Isn't charging people to use other carriers one of the big things that got Bell broken up? Google seems to tread in a of very dangerous waters. I wonder if they're just not aware of how close they are to being shutdown entirely, simply because so far they've always withstanded most legal challenges.
• specialist 7y ago
  Thank you for sharing your experiences, insights. #1 - Does anyone send test emails and measure delivery rates? As in send yourself a bunch of emails and see what happens. USPS and its major customers and vendors do this with physical mail. They measure stuff like UAA (undeliverable as addressed). FWIW, their Inspector General estimates 4.3% of mail was UAA in 2013. Report Number: MS-AR-14-006 https://www.uspsoig.gov/document/undeliverable-addressed-mail https://www.uspsoig.gov/document/undeliverable-addressed-mai... #2 - What is the responsibility, liability for email relays to treat everyone equally? For comparison, a US retailer has to accept US currency, but can (sometimes) turn away problematic clients. Is there anything like that for electronic exchanges, transactions?
• TheTruth1234 7y ago
  Fair play. I like my emails with pesto sauce, tuna, and cheese ... wicked combination.
• StreamBright 7y ago
  One more reason to de-Googlify my life. AWS has WorkMail I think it is time to try it out.
• no_wizard 7y ago
  I often see and hear about this issue a lot and I wonder how other email companies cracked it. I use FastMail for instance and never have this issue, but I know so many people who gave up on running their own mail servers at the small enterprise level because of stuff like this I often wonder how FastMail does not have these issues but others do. Is it a headers thing I wonder?
  • lolc 7y ago
    Fastmail is big compared to the operator in this article. Fastmail is probably big enough that some Gmail engineers will know how Fastmail's SMTP chatter looks. If you don't have volume you don't get that.
• JohnFen 7y ago
  Interesting... Like the author, I've been running my own mailserver for over a decade and am very conscientious about ensuring that no attackers use it as a spam relay. While the vast majority of the people I exchange email with don't use GMail at all, so it can take a while before I notice any issues with it, I did happen to notice that GMail was rejecting my outgoing email a couple of weeks ago. This week, I finally got around to trying to address the problem (it's not high priority because having GMail reject my emails isn't really a huge deal). ...and I found that it is working again without my changing anything. Weirdness abounds.
• imhelpingu 7y ago
  Have also run into this. Google is the quintessential evil tech corporation. We should all start avoiding their services when possible.
• rdlecler1 7y ago
  Ironically Google's spam always seems to go through...
• reedlaw 7y ago
  I'm quite happy with the ease of setting up and running a Mail-in-a-Box [1] instance. After seeing this article I did an experiment and while my hosted mail sent to a @gmail.com address did not bounce it went straight to the Spam folder. At least I can add other users to my email and enjoy private conversations. 1. https://mailinabox.email/ https://mailinabox.email/
• verisimilitudes 7y ago
  This also mirrors my experience, as I host my own email and it's solely for me. At one point, I couldn't successfully send any email to gmail, but then it changed for some reason and simply gets sent to spam folders instead. Perhaps it's entirely related to some people emailing me first that it gets through at all. I don't send much mail to gmail, though. Sans that, my only issue has been a mail server that uses Reverse DNS, which I don't have set up, and entirely ignores my email without it. I suppose I can understand this if some people get a great deal of spam, but requiring so much of this on an unencrypted message seems more like useless reassurances than anything. I'm not criticizing email for being unencrypted, but this seems more like another hoop to jump through than anything. Also of note, almost all of the spam I receive is from gmail addresses and I wouldn't be surprised if the invalid addresses that send messages demanding bitcoin are also from gmail, but with fake From fields.
• tssva 7y ago
  Regarding the issue with mailing lists for open source projects this isn't just an issue regarding Gmail. DKIM and DMARC cause no end of problems for mailing lists and it effects mail delivery to all major providers. Mailman, which most open source projects use for mailing lists, have developed work arounds to address some of the issues. Unfortunately my experience is that many projects run older versions that don't have these work arounds or if running newer versions they have not been enabled. Most likely because no one has revisited the configuration since initial deployment on an older version. After all they didn't start the project to spend their time being mailing list admins.
• cmsimike 7y ago
  Someone posted on reddit the other day about (seemingly) the exact same issue https://www.reddit.com/r/selfhosted/comments/bbrvlt/any_recommendations_for_a_reliable_smtp_service/ https://www.reddit.com/r/selfhosted/comments/bbrvlt/any_reco...
• npsimons 7y ago
  I'll just chip in with everyone else who is also self-hosting: I'm in a similar boat; been running email servers since before GMail existed. My personal one I've been running out of a home server closet since 2001. I've also done everything I can to guarantee I'm not running an open relay and not sending email unsolicited. Have been mostly lucky so far, but occasionally I will have people on mailing lists I manage (people I have met IRL and put them on the list to organize group meetings IRL) not get email. Used to be other stupid mail providers (AOL comes to mind), but these days it appears to be Google, sometimes. I've had this domain nearly twenty years and run email on it for that same amount of time. I'm not going to "just switch", especially to a privacy invading ad-spewing "alternative" that doesn't give me as much control. Fix your damn servers, Google.
  • antt 7y ago
    Google seems to have entered terminal senility. In my circle of friends googles search engine is pretty much useless, they have killed all the useful applications we used and now this. I can't help but think that we are seeing the google transition to late 90s microsoft now.
    • chezhead 7y ago
      What do you mean by "In my circle of friends googles search engine is pretty much useless"
      • antt 7y ago
        I mean that artists are unironically using bing as a better alternative for image searches, academics have gone back to tracking citations + libgen/scihub to find information and programmers are using duck duck go to debug their code. This is a huge problem since these people used to be the core of googles search user base.
• Felz 7y ago
  This article didn't mention whether or not their servers are attempting delivery using TLS/StartTLS, which is a good thing to check. I think Google penalizes email delivered insecurely.
• kazinator 7y ago
  Running your own e-mail server for receiving SMTP makes a lot of sense in many ways. I've been doing it for close to a decade myself. Sending SMTP yourself (directly, without an SMTP relay service) sets you up for trouble.
• deleted 7y ago
  [deleted]
• qope 7y ago
  Hacker news is fucked by google
• john37386 7y ago
  I'm in the same situation as many people here. Started as a small personal email projects many years ago just to learn. I really enjoyed it, reading all the specs and making it always better year after year. Dmarc felt like a total achievement. I started to host emails to many friends, small businesses and even a SaaS I developed. The subscription needs an email validation and I'm aware that the activation email ends up in the Spam folder for the new customers using Google emails. This activation email has everything from dkim, spf, dmarc, to unsubscribe link, full physical address of the business, etc and still I can't hit a good enough score. I was thinking to start using Google service to send the activation link and hosting my personal domains, but seeing that I am not alone, I will continue to improve my little email projects. Thanks all for cheering me up on this. I'm sure we can come up with a solution and I would be happy to help. When do we start?
• mullingitover 7y ago
  Gmail can be pretty aggressive with spam filtering, but I'm very happy with the bias toward flagging things as spam. Spam was such an infuriating problem, and lots of marketers would happily push the line if they didn't live in terror of being blacklisted by Google. If only Google could the legacy telephone networks like they run Gmail.
  • sbov 7y ago
    Flagging as spam is different. The OP is talking about rejecting it before it even gets to your inbox. If gmail is improperly flagging a message I get as spam, I can create a filter to never send it to spam. If they're rejecting it like the OP talks about, if I want to stay with gmail I have no way to get that email.
  • lolc 7y ago
    I get a fair amount of spam from Gmail accounts. abuse@gmail.com seems defunct to me because I never got an answer to my reports of unsolicited mail. I absolutely don't want them to run the phone network I'm using.
• deleted 7y ago
  [deleted]
• jboy55 7y ago
  If I recall my time at a main-sleaze email sender, some ISPs will send SMTP retry requests on random incoming email to see if there is a functioning SMTP server on the other side of the originator's email. Often times, spammers won't care about the email that's being sent, so won't respond, or in cases where they are hiding the origin, won't be able to respond.
• alanlovestea 7y ago
  I am in the process to detach Google products, somehow gmail is the only product I cannot find a working alternative.
• z3t4 7y ago
  I've also run a mail server for a long time. Yahoo and Hotmail are probably the most annoying, Yahoo because you can't do anything about it and Hotmail for blocking entire IP-ranges. I don't blame them though, preventing spam is very hard. Spammers always have spf,dkim,dmarc,reverse dns,signing, etc meanwhile many legitimate senders has none. I'ts a hard problem to solve. I've tried things like increasing the response time so the sender have to sit and wait, or used spam lists, but there are too many false positives. One easy solution is to make it illegal to send spam. Since my country made sending unsolicited e-mail an offense, spam decreased a lot. The second problem is hacked servers. Unless you are running a mail server, always block port 25 in the firewall. So at least the hackers wont be able to send spam on your behalf. Many ISP's already block port 25.
• gmailvictim 7y ago
  Add me to the victim list. I've been running my own mail server for 13 years and have to tell everyone I'm intending to email to check their spam as that's probably where my mail will be. I send a handful of mails a week probably. Almost always goes to the bin if it goes to gmail.
  • ponytech 7y ago
    Same here: I have to text my friends to tell them I sent them an email and check their spam. Nonsense.
• jasonvorhe 7y ago
  That's the price you're paying for a) self hosting and b) relying on decentralized protocols that don't have sensible abuse protections. People hosting their own servers enabled wide spread abuse due to misconfigurations. Because everyone could do it and because defaults were shit for decades, stuff like open relays were common. People defaulted to the wrong ports. Almost no one bothered to offer STARTTLS/Transport Encryption. Spam would have killed mail by now if it hadn't been for major players like Google, GMX, Hotmail/Outlook/etc. Back in the day, greylisting was commonly regarded as a best practice, leading to the impression that email is unreliable and prone to latencies. I'm sorry it's this difficult to host mail by yourself nowadays, but I'm happy to have a spam-free inbox every day and if this is the price for that, I'm sure about 1-2 billion people are willing to pay it. I'm quite astounded that there have been no updates to mail protocols in the last couple of years to at least mitigate the most common issues, but all I see are band-aids that are complex to setup and horrible to debug in case of issues.
• neilv 7y ago
  If ProtonMail would standardize a client protocol that other MUAs implement (not their "bridge" kludge), I'd start encouraging people to move there. I'm not interested in ProtonMail's encryption (and it's potentially a liability, attracting aggressive state action). I'm mainly interested in their apparent respect for the privacy of users' private communications. And also hoping that ProtonMail has a bit more reliable delivery than GMail. In any case, rising competition lifts all performance boats, or something like that.
• feanaro 7y ago
  Reading this thread made me realize the magnitude of this problem. It seems very sensible that small email server administrators should unite into a single effort in order to better publicize this issue, raise awareness and shame Google into compliance. Google seems to be suffocating the internet bit by bit on all fronts and it needs to be stopped.
• zzo38computer 7y ago
  I use my own SMTP server for receiving, but for sending I use the ISP's server (I think I have seem to have read somewhere that not only Google but some other services as well, will not accept mail from dynamic IP addresses). I have not had problems that I know of, so far. Please tell me of whatever mistake I may have made of such thing.
• alfiedotwtf 7y ago
  Imagine a phone carrier dropping your calls before the phone even rang on the other end. This is how I feel most of the time when I email people. Every new recipient I email, if I don't hear from them within 2 days, I have to contact them out of band to ask them to check their spam folder. The problem is usually Gmail's heavy filtering.
• tempestn 7y ago
  Better that what I've been seeing, which is Google just sending emails sent from my private domain to spam, with no bounce back or notification of any kind. This is despite the fact that I send the emails FROM gmail using their 'send as' feature. Nor does it seem to matter if I'm writing to a contact whom I've exchanged many emails with in both directions for years. If I send from my @gmail address they always go through. Send the same content from my domain and it's a crapshoot. Edit: And of course, I do have DKIM and SPF configured.
• jayalpha 7y ago
  Self hosting email is a pain in the butt. Try fastmail infomaniak.com (can also buy email for external domain) gandi.net (email included in domain)
  • jayalpha 7y ago
    PS: From the comments of the link "We run small hosting company, if any domain blocks our clients mails we block theirs and problem gets solved magically." :-)
• peterwwillis 7y ago
  Cue the *"but it's so easy to host your own email! Mine has been working fine for years!" Until this happens. Which it does very often, for lots of reasons that are out of your control.
• deleted 7y ago
  [deleted]
• deleted 7y ago
  [deleted]
• muppetman 7y ago
  I have a lot of success by having put my mailservers in the dnswl.org system.
• blunte 7y ago
  The black box is more mysterious to me. I had a client for whom I managed a G-Suite account with a custom domain name. The domain was not on a blacklist, and things generally worked well. One day a colleague and I discovered that he had not received some of my emails (intra-domain - me@example.com to him@example.com). This is all within the confines of Google. Google had flagged some messages as spam, and by what determination I could not fathom. The content seemed perfectly typical. I have had really pleasant experiences with G-Suite human support, at least in terms of the quality of interaction. But they could not answer why some intra-domain emails were being flagged as spam. I have suspicions that it would take a whole team of G engineers to maybe identify what bit of logic in their systems (incorrectly) marked some of the emails as spam. It seems the beast (automation) is just almost not under their control anymore.
• GnarfGnarf 7y ago
  I use non-Google email with SpamArrest, gives me complete control over whitelist/blacklist. Legitimate users always have a way of getting through.
• dangjc 7y ago
  I tried to set up a Discourse discussion forum for our neighborhood this year, and got stuck on setting up an email server that wouldn’t get blocked automatically by all major providers. Email is not a healthy ecosystem. You have to pay to send from one of the big trusted providers or else you get blocked just because your email ip address is untrusted by default. Yes fighting spam is important, but it’s shocking how much email providers rely on simple ip filters and trust levels rather than AI analysis of email content.
• angry_octet 7y ago
  Its like the deep learning has identified a feature it can use to determine spam: is this from the top 10,000 domains? No? It's spam. Spam from small domains might be pretty high as a category, but of course we don't want statistical judgements about categories to outweigh the merits of the individual. Maybe Google's algorithms have been watching too much Fox News.
• johnnyhead 7y ago
  If people stopped using G in such high percentage, then they would most likely need to adapt back to a more open internet, instead of trying to wall it off around their services. Reminds me of jabber.ccc.de that stopped providing new accounts because they felt they were ruining a federated system.
• derefnull 7y ago
  Thanks for writing this up, this mirrors my experience setting up a modern email server for personal & business use. I currently self-host for non-mission-critical email, use FastMail for business, and continue to use google apps for personal/mission-critical.
• richardriko 7y ago
  Does google have legal obligation to receive mails from 3rd party smtp servers ?
• strimp099 7y ago
  "Only when something like this happens you realize just how impossible it is to talk to someone on the modern internet without having Google somewhere in the middle." This says it all and is likely become a strategic imperative for Google along the way.
• vinay_ys 7y ago
  Does anyone here use encrypted and signed messages on a regular basis? How does gmail handle that on either sending or receiving ends? Is delivery rate better or worse if it is fully encrypted?
• ilaksh 7y ago
  To me this is more evidence that we definitely need public decentralized platforms that can replace Google and once we have them we need to eliminate the company. Private companies should not be allowed to monopolize platforms and control our lives.
• SourceParts 7y ago
  Gmail is rejecting valid content as spam and it never hits my spam folder. If I was not expecting an email as I was in this case, I would be totally oblivious. Something is up with the spam filter recently. They might have decrease the tolerance for spam and as a result a lot of legitimate traffic is being hit in the cross fire.
• Brittany_Lee 7y ago
  Google eats everything
• zahllos 7y ago
  I've also hosted my own email for several years now, on several domains. The one thing I have that does not appear to be listed is a DMARC policy. As the comments on the blog post suggest, configuring DMARC works. I have no problem delivering to either Gmail or Outlook. I don't think it's unreasonable to be strict regarding DMARC delivery. My MTA has a fairly strict SPF configuration - any email with an invalid spf result is rejected. This can come about because a legitimate company has misconfigured their spf records (happened twice in all the years I have hosted, discussions via postmaster@ helped them configure their dns correctly), but 99.999% of the time it is a spammer. What is worse is that rejecting email for domains without any SPF records can still result in valid email being lost, in 2019. In this specific case, I don't think Google are "being evil". They're trying to reduce spam in the email ecosystem and they're doing it by using standards they themselves adhere to (Gmail send me reports of dmarc statistics each day google domains receive email from my box). On the other hand, I do of course support either self hosting, or using another provider so as to ensure we do not end up with a Gmail monopoly. If I did not self host, I would find another provider like (but may not) Fastmail, Posteo etc (I would have to seriously review the options, which I haven't done).