5 ms·

I used to run my own mail server, and that very nearly cost me a career-changing job at Microsoft. My reply to a hiring manager for a team I really wanted to wo

I used to run my own mail server, and that very nearly cost me a career-changing job at Microsoft. My reply to a hiring manager for a team I really wanted to work on went into Microsoft's bit bucket because (I'm guessing) their spam detectors didn't like my own domain and my own instance of Exim sending out my emails. When a recruiter (thankfully) followed up and scheduled me for on-site interviews, that manager mentioned that he never got a reply when he emailed me. That was the point where I realized that email had gone the way of fiefdoms, and if I wanted really important emails to make it through, I'd need to choose which castle I wanted to handle my email.

tzs 6y ago
Wait...your mail was a reply to a mail they sent to you, and it went to the bit bucket? Their spam handling is not sophisticated enough to special case replies to mails that they have sent?
- travbrack 6y ago
  Do other spam filters actually check every incoming email against a database of outgoing emails?
  - lisper 6y ago
    Mine does. And I wrote it myself.
    - eru 6y ago
      You could use a Bloom filter to save on memory. And perhaps empty it once a year, if necessary. (Ie so that replies to emails older than a year don't rank better than random emails.)
      - lisper 6y ago
        I just throw all the subject lines into a database. The data volume is not even remotely close to the limits of a modern DB.
        eru 6y ago
        Oh, I was thinking of how eg GMail could do it. It depends a bit on volume of spam compared to legitimate emails. GMail knows about all of your past outgoing messages anyway, but the bloom filter would allow to quickly drop spam. Google is already using Bloom filters to eg check locally in one service whether a cache in a different service is likely to be able to answer a query.
        deleted 6y ago
        [deleted]
- luckylion 6y ago
  Google does this as well - neither a history of emailing back and forth nor a same-day reply will keep incoming email safe from being classified as spam if it's being sent by an ordinary mail server. Hell, Google even manages to classify emails as spam that are sent from one address on a GSuite account to another address on the same account.
  - eitland 6y ago
    I think it is time we realize that Google in its current form is broken in many ways. This is just one example. Others include: - insisting on captchas,and not just one, even when I'm logged in with my >10 year old, paid for account with two-factor enabled - not being able to detect flag attacks against legitimate accounts (as seen on the front page today) - killing reader, trying to force everyone to use a single identify, then at the same time name their otherwise brilliant social network the same thing as the hated identity solition - then killing said social network - not being able to even keep the quality on their flagship user facing product (for us who knew it 10 and 15 years ago what we see today is a mockery.) I hope someone here can submit applications for next batch of YC: - like Google search in 2008. I'll happily turn off your adblocker for you and be glad to see simple, relevant text ads. Or pay up to USD5 monthly if thats better (more if necessary and if the quality is good enough, although at some point I'd probably need to expense it.) - like Google+ but not owned by Google - like Google Reader but not owned by Google And, probably also (unrelated to Google): - like stackoverflow but focused on answering questions, not playing political and nitpicking games - like WhatsApp before they were acquired (and with a strong guarantee that this new company will stay independent)
    - sakisv 6y ago
      Some possible alternatives you could try: For search engine I'd suggest duckduckgo, it's far from perfect but I don't see any other viable alternative, if anyone knows anything please let us know. For messaging I would suggest Signal. The problem there is that most people don't know of it and they won't ditch whatsapp to go on something that their friends won't be on. That's the same problem that a new social network is going to face. I think it's safe to say that if even Google with all their capital couldn't make it work, then it's going to be extremely difficult to find investors to bet on a random guy's idea. Facebook with all the shit it has been and done has almost 2bn users, how are we going to convince even 5% of them to abandon it for something new without anyone they know there?
- anoncake 6y ago
  Microsoft's spam filter is awfully strict (the one for outlook.com at least but I don't think the one for its employees is that different)
  - tzs 6y ago
    From what I've seen, outlook.com does seem to have some special case for mail from people you've sent to. I did some experiments a few months ago, and again a week or two ago with mail delivery, and outlook.com was one of the targets. Here's what I did. I have a domain, which I'll refer to as @mine in the following. It's a .net domain that I've had for 20+ years. It has never been used by anyone other than me. Before I had it, it was owned by my employer which had bought it for me as a gift and transferred it to me. They were the first to ever register it, but never used it for anything. @mine has never sent spam, or any commercial email at all except for either replies to received commercial email or to contact businesses I was a customer of or wanted to be a customer of at their support or sales addresses. In the early 2000s I think spammers occasionally forged @mine emails, back before SPF was a thing. I added an SPF record when those became normal, and later DKIM and DMARC records. The various online checking tools tell me that these are all set up correctly. I've never had a hit checking for @mine with the various online blacklist checking tools. I did some experiments a few months ago, when @mine was handled by a server of my own, running on a VM at Rackspace. All the test messages were short, with no URLs or images, mentions of money, or anything else suspicious. Thing's like "Bob's birthday party will be Saturday instead of Sunday. Can you make it?"...i.e., innocuous innocent things that ordinary people might send to other ordinary people. @mine -> gmail.com: went to inbox. It shows that SPF, DKIM, and DMARC all pass. I'm not sure that this was a valid test though, because I'm pretty sure that Google knows that me@mine is the same person as me@gmail, so they might give it preferential treatment. @mine -> yahoo.com: went to spam. I marked it as not spam, and sent another one. That and subsequent messages went to inbox. @mine -> outlook.com: went to spam. I marked it as not spam, and sent another one. Went to spam. Repeat the mark and send thing several times. Still goes straight to spam. A couple months or so later, the Linux distro I ran my server on was rapidly approaching EOL for long term support, and I moved @mine to Fastnet.com and cancelled my Rackspace account. (The only other thing @mine was a very low volume web server...I moved that to Amazon Lightsail. $3.50/month for that + $3.61/month for Fastmail is less than 1/3 of what the cheapest Rackspace VM cost (and I was on a grandfathered plan there that was cheaper than any of the current plans). I repeated the experiment with @mine now at Fastmail, to see if the problem at outlook.com was that I was running my own server at a public cloud. @mine ->gmail and yahoo still went to inbox, just as they did at the end of the first experiment. @mine -> outlook.com: still to spam. A few more cycles of mark not spam and try again did not change this. I own another domain, my ham radio call sign in .us. I grabbed that when I got my license, and have never done anything with it. It just had whatever standard parking page Namecheap uses and whatever standard mail handling they provide when you register a domain there and don't do anything with it. I switched @ham mail handling over the Fastmail, and tried the experiment again. Same results as the first @mine experiment. Gmail never went to spam, yahoo went to spam once, and then to inbox after I told it that one was not spam. Outlook straight to spam despite repeated attempts to train it otherwise. I also tried marking @ham as a "safe sender" in outlook, which is supposed to bypass the spam checking. Nope. Still straight to spam. Some Googling turned up reports on MS support forums of "safe sender" not working, and it appears that this has been a long term problem. (I also found that sometimes my "safe sender" list would disappear. Outlook.com, at least via the web interface, seems rather flaky). (I saw some other posts that suggested that using rules was also insufficient...apparently rules processing takes place after spam processing?) Finally, I sent a mail from outlook.com to @ham. After that, mail from @ham went to inbox. It would be nice to try now with a third new domain, this time sending from outlook to @third before ever trying to send to outlook, and then seeing what happens to the first mail from @third to outlook. Actually, it would be best to also do this with @fourth, with the first mail from @third being a reply to the mail from outlook, and the first mail from @fourth not being a reply. But I don't have any other domains. Maybe the next time I see a really cheap domain promotion at Namecheap I'll grab a couple and try this. They've got .xyz for $1/year right now. Maybe if I get bored enough with the COVID-19 "stay at home" I'll grab a couple and give it a try.
maheart 6y ago
Do you remember what sort of validation you had setup (if any)? e.g. SPF, DKIM, IP from reputable block, etc. I'm curious to what degree this can potentially be mitigated.
- 0ld 6y ago
  spf, dkim and clean ip were enough for my opensmtpd to deliver mail from my domain into gmail and outlook inboxes
  - delusional 6y ago
    i have similar experiences. clean ip (actually from a hetzner block, but not on any lists), and dkim. I have never dropped an outgoing mail.
  - danieldk 6y ago
    The problem is that you are really dependent in what other people in your address range do. One malicious user in the same DC could reduce the reputation of an address range and then you get mail delivery problems. I'd rather pay someone to sort out such issues for me. The $5 p/m or so I pay Fastmail monthly are far cheaper than my own time.
DeathArrow 6y ago
I use Gmail for personal mail and Yandex mail for my domains. I tried SendGrid but when I tested with Gmail, Outlook.com and Yahoo, most mails got into spam folder. It's best to test when you set up email, whether you use your own server or use a provider. Most of the times, sending mail from your own server will guarantee your mail gets into spam folder. I can highly recommend Yandex as a mail provider.
- 0ld 6y ago
  > I can highly recommend Yandex as a mail provider. nice try, comrade. but no

tzs 6y ago

Wait...your mail was a reply to a mail they sent to you, and it went to the bit bucket? Their spam handling is not sophisticated enough to special case replies to mails that they have sent?

travbrack 6y ago
Do other spam filters actually check every incoming email against a database of outgoing emails?
- lisper 6y ago
  Mine does. And I wrote it myself.
  - eru 6y ago
    You could use a Bloom filter to save on memory. And perhaps empty it once a year, if necessary. (Ie so that replies to emails older than a year don't rank better than random emails.)
    - lisper 6y ago
      I just throw all the subject lines into a database. The data volume is not even remotely close to the limits of a modern DB.
      - eru 6y ago
        Oh, I was thinking of how eg GMail could do it. It depends a bit on volume of spam compared to legitimate emails. GMail knows about all of your past outgoing messages anyway, but the bloom filter would allow to quickly drop spam. Google is already using Bloom filters to eg check locally in one service whether a cache in a different service is likely to be able to answer a query.
        deleted 6y ago
        [deleted]
luckylion 6y ago
Google does this as well - neither a history of emailing back and forth nor a same-day reply will keep incoming email safe from being classified as spam if it's being sent by an ordinary mail server. Hell, Google even manages to classify emails as spam that are sent from one address on a GSuite account to another address on the same account.
- eitland 6y ago
  I think it is time we realize that Google in its current form is broken in many ways. This is just one example. Others include: - insisting on captchas,and not just one, even when I'm logged in with my >10 year old, paid for account with two-factor enabled - not being able to detect flag attacks against legitimate accounts (as seen on the front page today) - killing reader, trying to force everyone to use a single identify, then at the same time name their otherwise brilliant social network the same thing as the hated identity solition - then killing said social network - not being able to even keep the quality on their flagship user facing product (for us who knew it 10 and 15 years ago what we see today is a mockery.) I hope someone here can submit applications for next batch of YC: - like Google search in 2008. I'll happily turn off your adblocker for you and be glad to see simple, relevant text ads. Or pay up to USD5 monthly if thats better (more if necessary and if the quality is good enough, although at some point I'd probably need to expense it.) - like Google+ but not owned by Google - like Google Reader but not owned by Google And, probably also (unrelated to Google): - like stackoverflow but focused on answering questions, not playing political and nitpicking games - like WhatsApp before they were acquired (and with a strong guarantee that this new company will stay independent)
  - sakisv 6y ago
    Some possible alternatives you could try: For search engine I'd suggest duckduckgo, it's far from perfect but I don't see any other viable alternative, if anyone knows anything please let us know. For messaging I would suggest Signal. The problem there is that most people don't know of it and they won't ditch whatsapp to go on something that their friends won't be on. That's the same problem that a new social network is going to face. I think it's safe to say that if even Google with all their capital couldn't make it work, then it's going to be extremely difficult to find investors to bet on a random guy's idea. Facebook with all the shit it has been and done has almost 2bn users, how are we going to convince even 5% of them to abandon it for something new without anyone they know there?
anoncake 6y ago
Microsoft's spam filter is awfully strict (the one for outlook.com at least but I don't think the one for its employees is that different)
- tzs 6y ago
  From what I've seen, outlook.com does seem to have some special case for mail from people you've sent to. I did some experiments a few months ago, and again a week or two ago with mail delivery, and outlook.com was one of the targets. Here's what I did. I have a domain, which I'll refer to as @mine in the following. It's a .net domain that I've had for 20+ years. It has never been used by anyone other than me. Before I had it, it was owned by my employer which had bought it for me as a gift and transferred it to me. They were the first to ever register it, but never used it for anything. @mine has never sent spam, or any commercial email at all except for either replies to received commercial email or to contact businesses I was a customer of or wanted to be a customer of at their support or sales addresses. In the early 2000s I think spammers occasionally forged @mine emails, back before SPF was a thing. I added an SPF record when those became normal, and later DKIM and DMARC records. The various online checking tools tell me that these are all set up correctly. I've never had a hit checking for @mine with the various online blacklist checking tools. I did some experiments a few months ago, when @mine was handled by a server of my own, running on a VM at Rackspace. All the test messages were short, with no URLs or images, mentions of money, or anything else suspicious. Thing's like "Bob's birthday party will be Saturday instead of Sunday. Can you make it?"...i.e., innocuous innocent things that ordinary people might send to other ordinary people. @mine -> gmail.com: went to inbox. It shows that SPF, DKIM, and DMARC all pass. I'm not sure that this was a valid test though, because I'm pretty sure that Google knows that me@mine is the same person as me@gmail, so they might give it preferential treatment. @mine -> yahoo.com: went to spam. I marked it as not spam, and sent another one. That and subsequent messages went to inbox. @mine -> outlook.com: went to spam. I marked it as not spam, and sent another one. Went to spam. Repeat the mark and send thing several times. Still goes straight to spam. A couple months or so later, the Linux distro I ran my server on was rapidly approaching EOL for long term support, and I moved @mine to Fastnet.com and cancelled my Rackspace account. (The only other thing @mine was a very low volume web server...I moved that to Amazon Lightsail. $3.50/month for that + $3.61/month for Fastmail is less than 1/3 of what the cheapest Rackspace VM cost (and I was on a grandfathered plan there that was cheaper than any of the current plans). I repeated the experiment with @mine now at Fastmail, to see if the problem at outlook.com was that I was running my own server at a public cloud. @mine ->gmail and yahoo still went to inbox, just as they did at the end of the first experiment. @mine -> outlook.com: still to spam. A few more cycles of mark not spam and try again did not change this. I own another domain, my ham radio call sign in .us. I grabbed that when I got my license, and have never done anything with it. It just had whatever standard parking page Namecheap uses and whatever standard mail handling they provide when you register a domain there and don't do anything with it. I switched @ham mail handling over the Fastmail, and tried the experiment again. Same results as the first @mine experiment. Gmail never went to spam, yahoo went to spam once, and then to inbox after I told it that one was not spam. Outlook straight to spam despite repeated attempts to train it otherwise. I also tried marking @ham as a "safe sender" in outlook, which is supposed to bypass the spam checking. Nope. Still straight to spam. Some Googling turned up reports on MS support forums of "safe sender" not working, and it appears that this has been a long term problem. (I also found that sometimes my "safe sender" list would disappear. Outlook.com, at least via the web interface, seems rather flaky). (I saw some other posts that suggested that using rules was also insufficient...apparently rules processing takes place after spam processing?) Finally, I sent a mail from outlook.com to @ham. After that, mail from @ham went to inbox. It would be nice to try now with a third new domain, this time sending from outlook to @third before ever trying to send to outlook, and then seeing what happens to the first mail from @third to outlook. Actually, it would be best to also do this with @fourth, with the first mail from @third being a reply to the mail from outlook, and the first mail from @fourth not being a reply. But I don't have any other domains. Maybe the next time I see a really cheap domain promotion at Namecheap I'll grab a couple and try this. They've got .xyz for $1/year right now. Maybe if I get bored enough with the COVID-19 "stay at home" I'll grab a couple and give it a try.

maheart 6y ago

Do you remember what sort of validation you had setup (if any)? e.g. SPF, DKIM, IP from reputable block, etc. I'm curious to what degree this can potentially be mitigated.

0ld 6y ago
spf, dkim and clean ip were enough for my opensmtpd to deliver mail from my domain into gmail and outlook inboxes
- delusional 6y ago
  i have similar experiences. clean ip (actually from a hetzner block, but not on any lists), and dkim. I have never dropped an outgoing mail.
- danieldk 6y ago
  The problem is that you are really dependent in what other people in your address range do. One malicious user in the same DC could reduce the reputation of an address range and then you get mail delivery problems. I'd rather pay someone to sort out such issues for me. The $5 p/m or so I pay Fastmail monthly are far cheaper than my own time.

DeathArrow 6y ago

I use Gmail for personal mail and Yandex mail for my domains. I tried SendGrid but when I tested with Gmail, Outlook.com and Yahoo, most mails got into spam folder. It's best to test when you set up email, whether you use your own server or use a provider. Most of the times, sending mail from your own server will guarantee your mail gets into spam folder. I can highly recommend Yandex as a mail provider.

0ld 6y ago
> I can highly recommend Yandex as a mail provider. nice try, comrade. but no