22 ms·

The problem is that spam was/is so bad that extreme measures were taken to curb it. There are all kinds of invisible forces that you abutt that can be difficult

The problem is that spam was/is so bad that extreme measures were taken to curb it. There are all kinds of invisible forces that you abutt that can be difficult to figure out, such as IP blacklists and the like. And even if you set everything up properly and host your email with a responsible host, Microsoft will still mark your mail as spam. I host my own email server with Vultr on an OpenBSD VM using OpenSMTPD and Dovecot, relaying all outbound mail through SMTP2Go (their free tier more than meets my needs). I have all of the necessary DNS entries set to mark my mail as legit, and I sign all outgoing mail using strong 2048-bit RSA keys. Thus far, I'm able to send mail and not have it marked as spam (at least to everyone that I've corresponded with thus far). It was a lot of work to get to that point, but not terrible.

rhizome31 4y ago
> relaying all outbound mail through SMTP2Go So it's not an entirely self-hosted solution, is it?
- Veen 4y ago
  No, but it's quite difficult to have email reliably and consistently delivered to Gmail and other major email providers without sending it via a relay. The relay provider is in the business of maintaining IP addresses with good reputations that aren't blocked by spam lists etc. If you can find and keep a reputable IP address, then you're fine, but it's usually easier to pay someone who does that for a living—you have no guarantee that the IP address assigned to you by Digital Ocean or whoever wasn't used for spamming at some point.
  - cube00 4y ago
    Digital Ocean has an extremely poor reputation over a long period to the point where their droplets are blocked on mass in many places now [1] Even my local ISP refuses mail from them. [1]: https://discourse.mailinabox.email/t/digital-ocean-ips-being-blacklisted-by-more-and-more-esps/8502 https://discourse.mailinabox.email/t/digital-ocean-ips-being...
    - teh_klev 4y ago
      Really sorry, I don't normally nit pick spelling and grammar, but it's "en masse" rather than "on mass".
    - bombcar 4y ago
      Yeah, lots of places just straight up block entire IP ranges, such as anywhere you can get a VM for cheap/free, or residential IP ranges, etc.
    - zcdziura 4y ago
      This is also why I went with Vultr as my server host. They block port 25 by default and make customers file a support ticket with them to unblock that port. They also require your account be active for at least a month and be using their service in good standing during that time. Wasn't an instant process, but was simple enough to accomplish in the end.
tonmoy 4y ago
Microsoft had marked an email from a professor from an vt.edu domain email address as spam causing me to miss an interview for a PhD funding.
- codegeek 4y ago
  Microsoft is especially notorious for flagging legit emails as spam if they are not from one of the regular providers.
  - cube00 4y ago
    Flagging if you're lucky, they outright 550 refused my mail until I joined their sender program and applied to have my domain unblocked. Then they proceeded to gaslight me claiming my mail was never blocked even after I forwarded their own error messages and IDs back to them.
    - codegeek 4y ago
      yea unfortunately I have seen those as well. It is ridiculous at times.
    - anonydsfsfs 4y ago
      You got a 550? Lucky! When I worked at a non-profit that hosted our own email server, we had many instances where Microsoft would /dev/null our newsletter e-mails. Their servers would give a 250 indicating acceptance, but the e-mail was nowhere to be found (and yes, we checked the spam folder).
  - donmcronald 4y ago
    That's nothing compared to the joy of dealing with legit emails that are flagged as high confidence phishing.
  - maestroia 4y ago
    Regular provider == (Microsoft 365 || an Exchange Server)
- 4y ago
aaron_m04 4y ago
I have this part: > I host my own email server with Vultr on an OpenBSD VM using OpenSMTPD and Dovecot But with outgoing mail being relayed internally to dkimproxy which signs it before being relayed back to OpenSMTPD for delivery to the other email server. I had to set up SPF and DKIM DNS records, and one time I had to request that my IP be removed from the Abusix blacklist. Other than that, it's pretty rare for my emails to be marked as spam. Outlook 365 seems to do it much more often than Gmail though.
- zcdziura 4y ago
  That's very interesting. I never thought to relay mail internally to dkimproxy. I'll have to give that a shot. I like the idea of hosting the entire solution myself and not relying on any 3rd party solutions, but relaying through SMTP2Go was the only thing that I tried that actually solved the problem. Perhaps this will offer a good solution! Thanks!
  - wahern 4y ago
    I also use the dkimproxy package, but there's now a third-party OpenSMTPd module that can sign messages in-line.[1] I've always found dkimproxy setup a little confusing compared to a built-in/in-line solution. I might try to switch to the module during the OpenBSD 7.1 upgrade process. [1] I think this is the one I had I mind, though I didn't realize it was already in ports: https://cvsweb.openbsd.org/ports/mail/opensmtpd-filters/dkimsign/ https://cvsweb.openbsd.org/ports/mail/opensmtpd-filters/dkim...
    - throw0101a 4y ago
      If you run a mailing list you generally have to worry about ARC (re-signing 'chain of custody') in addition to DKIM: * https://en.wikipedia.org/wiki/Authenticated_Received_Chain https://en.wikipedia.org/wiki/Authenticated_Received_Chain I've found ARC to fiddle some to get going than ARC.
plainnoodles 4y ago
> There are all kinds of invisible forces that you abutt that can be difficult to figure out This was my main experience, and all I did was try to set up the ability to simply send emails to myself (gmail) (and no-one else). Things like: this script crashed, or btrfs scrub finished + scrub results, and similar. The first thing I tried was just setting up a VM with postfix running on it locally with my residential ISP. I don't even remember what the error was for this scenario, but it was just totally dead in the water. Absolutely zero mail delivery. I think I eventually figured out it's because google defers to spamhaus, and spamhaus says residential IPs = hard no. That next thing I tried, and what I ended up doing, was writing a docker container that just runs an SSH port forward to jump from my local network to a digitalocean host, which is where another docker container runs postfix. I had done this bit once before, and I tried to just set up DKIM (since DKIM was, to my reading, basically bulletproof - why bother with SPF when you have real cryptographic identity assurance?). This led to weird error messages from google about my IP having a super low reputation. This was something I'd been worried about so I spent a bit of time trying to cycle my IP. But I eventually figured out it was just a bad error message and setting up SPF suddenly made my emails start delivering. My main ongoing issue is that I had to add all my sending addresses (things my internalhostnamehere@myrealdomain.com) to my contacts in gmail, otherwise there was like a 50% chance they'd just go to spam. I've been running this setup for about a year and it's still a coin toss whether emails will come through fine, or if they'll say "this would've gone to spam but it's in your contacts". When that happens, I check the DKIM and SPF status in "original message" in gmail, and gmail itself says they both passed. Absurd tbh. For my "not self-hosted but better than letting google own my digital identity" solution, since I use apple icloud+ or whatever it's called, I set up the SPF stuff to let me send+receive email from my custom domain, so while icloud could still scan my mail, at least if I get banned, I still own the actual domain and could move somewhere else.
- miohtama 4y ago
  Even if one setups everything by the book (SPF, DKIM, DNS.) etc. No one at @outlook.com will receive email, based on my experience. Thus, it does not work well if email is important for business-to-business use. Outlook and Gmail are basically having opaque rules who can receive email and there is no process to get “whitelisted” on these receivers.
leros 4y ago
This is the answer. Blocked emails happen for random reasons and fixing them is a black art that involves talking to ISPs and stuff. It's really too much for an average person to handle. At work we've had issues with email delivery due to things like outdated IP block lists at some random ISP four hops away, only impacting deliverability when mail gets routed through that part of the web.
- paulmd 4y ago
  I have an email address with "spam" in the name (this is through gmail) and lately I've had all kinds of problems with emails to it disappearing - I've had to call several places and have them change my email because I can't log in and the reset emails don't ever show up... but changing to myfirst.mylast works fine. I've run into this with both Sam's Club and Speedway Rewards. Only thing I can think of is that some outbound mail service they're using is dropping them, or some relay in the middle is dropping them... I can see where the word "spam" would be a keyword you might use, but I've had this email address for 15 years now and it's only been a problem in the last few years.
post-it 4y ago
Prediction: Any distributed social media (like Mastodon) that gains mainstream popularity will share the same fate. Sure, you'll be able to host your own Mastodon instance, but 99% of people will be on the top 10 hosts and they won't peer with you. I think the only way to make distributed social media practical is to have an extremely inexpensive turnkey self-hosting solution for the average person. A Chromecast-like device that they plug into their TV that backs up all their photos, plays music, and also hosts a Mastodon instance. Some kind of very friendly backup solution where you make an "emergency contacts" list, and the device encrypts all of your data and stores it on your emergency contacts' devices as a backup, and vice-versa.
- mypalmike 4y ago
  On Mastodon, I believe it's currently somewhat backwards from this. The largest instances are filled with Japanese anime porn, and the smaller instances end up blacklisting them.
  - rootusrootus 4y ago
    Anecdotally, this has happened every time I've set up any kind of social media instance / discussion forum / BBS (back in the day!) / whatever. It immediately gets consumed by people who use it to host porn, and then all the intended users leave.
    - derefr 4y ago
      Have you considered creating a discussion platform where people can't post images, URLs / things that would be URLs if you added a URL scheme to them, ASCII-armored baseN-encoded anything, etc? For 99% of the discussion you want on the platform, text is all you need. For spammers and people who want to host porn, text alone is useless.
      - sodality2 4y ago
        Plenty of spammers rely on text, though. A good chunk of my spam folder is text only.
deleted 4y ago
[deleted]
tzs 4y ago
> And even if you set everything up properly and host your email with a responsible host, Microsoft will still mark your mail as spam. I did some experiments back when I ran my own mail. Sending from my mail server to my Microsoft account it not only marked everything as spam, it continued marking everything as spam after I marked a bunch of them as not spam. After that, I tried also answering several of them and composing several new mails to send to my non-Microsoft email to see if Microsoft's spam system was smart enough to figure out that if I'm actively corresponding with someone their incoming mail should not be marked as spam. It was not smart enough. Then I tried whitelisting. Nope, still spam.
wpietri 4y ago
Yup. Spam is the root problem. With an enormous amount of complexity between that and the mail admin's day to day experience. I hosted my own mail for more than 20 years. A couple years back I just got tired of trying to solve deliverability puzzles, plus the fears that deliverability issues generate. (E.g., "Did that potential employer get my email about the job?") Especially since some of the puzzles are not solvable, like why GMail does what it does. I even had friends at Google, and I still couldn't find out why GMail occasionally didn't like my server. And arguably, that's the right choice for them, as the more spammers know about how they work, the worse it is for Google staff and GMail users. For me, switching to Fastmail hosting was a big win. It's not like I'm out of technical challenges to solve, but I get to apply that to things where the upside is greater than, "The thing everybody expects to work still works."
- clairity 4y ago
  the spam problem advantages google, as your own story illustrates, so it's unlikely they'd really want to help solve deliverability/spam issues systemicly. making personal email hosting more difficult means they have a chance to capture your email data streams via gmail. whether you switch there or not, it creates a pressure for most to aggregate on gmail, which means they can see most email exchanges.
  - wpietri 4y ago
    For sure. Good spam filtering was one big reason for people to switch to GMail. And a lot of people who gave up hosting their own email have switched to Gmail as well. I'm sure this doesn't rise to the level of conspiracy, but there's little incentive for them to fix the broader problems.
- azinman2 4y ago
  My issue with fast mail et all is storage is so unnecessarily expensive. I have many gigs of email that I don’t want to lose, but I also don’t want to pay many tens of dollars/month to host it.
  - 4y ago
elorant 4y ago
Any chance you'll provide a detailed write-up of your experience with tips and whatnot?
inetknght 4y ago
> The problem is that spam was/is so bad that extreme measures were taken to curb it. The problem with spam is that there's no real legal recourse for spam. If it's in your own country then maybe. But outside of your country? Well the easiest thing to do is to IP block and the next best thing to do (when IP block isn't an option) is to use some sort of "smart detection" to put spam into a special box labeled "spam". There's no deterrence and literally no criminal prosecution for spam.
jjav 4y ago
Yes to the OP, you most definitely can host your own email fully. Many of us do it. If you have any interest in the topic, either due to the fun of managing the servers and learning something along the way or due to the moral high ground of supporting decentralization above proprietary walled gardens, do it! Ignore the naysayers, if you're interested you can do it. Will some emails very occasionally end up in the spam folder of a recipient? I mean, yes, but that is true of everything. You can end up in spam folder sending from Microsoft Office mail to gmail or vice versa. Heck, every now and then an email from my manager will end up in my spam folder in gmail even though he's emailing me from gmail to gmail, both of us in the same corporate gsuite account! So on average, once you set everything up correctly, your deliverability will be as good as gmail to gmail, which is to say not 100% perfect but no worse than any other solution. And you'll be in control of your email infrastructure and address. No longer will google/microsoft/apple/yahoo be able to cut you off all your accounts on the whim an AI gone bad. The parent post mentions a useful safety valve to know about if you're worried about deliverability and want to take baby steps to get there. You can always, either selectively or wholesale, use a commercial relay for outbound mail from your email server. Some have free tiers that are plenty for personal/family use. Personally I don't use any third party relay, I deliver to everywhere from my own infrastructure. No issues.
- leephillips 4y ago
  Me too, and this is my experience as well. In the rare event that I find out that someone isn’t getting my emails, I tell them that they should complain to their provider or use a different one. I’m no longer willing to jump through hoops so that hotmail delivers my email.
throw10920 4y ago
> The problem is that spam was/is so bad that extreme measures were taken to curb it. Man, and there's such an easy solution, too - just use Hashcash[1] (invented in 1997) and 90%+ of spam disappears overnight (if not more, depending on how high you set the difficulty). Well, ok, "easy" in the sense that We Have An Algorithm For This - it'd still be hard to get email clients/servers to agree on a protocol... [1] https://en.wikipedia.org/wiki/Hashcash https://en.wikipedia.org/wiki/Hashcash
- throw0101a 4y ago
  If there's a compromised machine it will be the victims paying the cost in energy bills for spammer's nefariously installed malware to send garbage.
  - throw10920 4y ago
    If there's a compromised machine, the scammer can drain the victim's bank accounts and cost them far more than an electricity bill, and/or mine cryptocurrency directly. Regardless, their spam-sending rate will still be significantly decreased. So, this argument is completely invalid.
    - throw0101a 4y ago
      > If there's a compromised machine, the scammer can drain the victim's bank accounts […] Not if the machine is a server and was gotten into via (e.g.) a bug in a web app. I don't know about you, but I don't keep my bank account information on the LAMP systems I sysadmin.
      - throw10920 4y ago
        That part of the post was specifically about consumer devices. You missed the rest: > and/or mine cryptocurrency directly. Regardless, their spam-sending rate will still be significantly decreased Your argument remains invalid.
- dTal 4y ago
derefr 4y ago
Is there such a service that will tell me the reputation of an email domain, i.e. whether mail originating at that domain would be likely to be treated as definitely spam or not? (I don't really care about "no reputation"; I want to know if a domain has known bad reputation.) I feel like, if there was such a service, it would be pretty useful to use it to prevent account registrations on other services, from users whose email addresses have domains with bad reputations. After all, they'd very likely just be registering with the intent of using the service to send or post spam in some way.
- TZubiri 4y ago
  multirbl.valli.org Contains blacklists on the domain level, also on the ip block and AS level.
  - derefr 4y ago
    Possibly interesting... but these are rules about outgoing SMTP servers (MSAs), yes? How much of a relation does the outgoing SMTP server for a domain have to the canonical set of receiving SMTP servers (MTAs) for the domain held in the domain's DNS MX record? These can certainly be one and the same server; but it's not a requirement. So how often are they in practice? Especially for people actively trying to evade these sorts of RBLs?
patrck 4y ago
Also, one should subscribe to the mailop mailing list, which serves as a Distant Early Warning line for email deliverability issues (ie. like NANOG for netops issues). https://www.mailop.org/best-practices/ https://www.mailop.org/best-practices/
raxxorraxor 4y ago
Spam got significantly worse but this is also an chance to curb the federalization of mail by large companies. Of course they would like you to use a Microsoft or Google account to send mail.
tinroofrusted 4y ago
zddziura, I want to say a big thank you, thank you, thank you for pointing me to SMTP2Go. I have been trying to get my DMARC and DKIM email woes solved for months, but couldn't get it figured out. When I read your post I signed up with SMTP2Go at the free tier and I had a 100% Mailgenius score in less than an hour after I set it up. So awesome! No more big yellow warning boxes in Gmail when receiving mail from my own domain! Yea!!!

rhizome31 4y ago

> relaying all outbound mail through SMTP2Go So it's not an entirely self-hosted solution, is it?

Veen 4y ago
No, but it's quite difficult to have email reliably and consistently delivered to Gmail and other major email providers without sending it via a relay. The relay provider is in the business of maintaining IP addresses with good reputations that aren't blocked by spam lists etc. If you can find and keep a reputable IP address, then you're fine, but it's usually easier to pay someone who does that for a living—you have no guarantee that the IP address assigned to you by Digital Ocean or whoever wasn't used for spamming at some point.
- cube00 4y ago
  Digital Ocean has an extremely poor reputation over a long period to the point where their droplets are blocked on mass in many places now [1] Even my local ISP refuses mail from them. [1]: https://discourse.mailinabox.email/t/digital-ocean-ips-being-blacklisted-by-more-and-more-esps/8502 https://discourse.mailinabox.email/t/digital-ocean-ips-being...
  - teh_klev 4y ago
    Really sorry, I don't normally nit pick spelling and grammar, but it's "en masse" rather than "on mass".
  - bombcar 4y ago
    Yeah, lots of places just straight up block entire IP ranges, such as anywhere you can get a VM for cheap/free, or residential IP ranges, etc.
  - zcdziura 4y ago
    This is also why I went with Vultr as my server host. They block port 25 by default and make customers file a support ticket with them to unblock that port. They also require your account be active for at least a month and be using their service in good standing during that time. Wasn't an instant process, but was simple enough to accomplish in the end.

tonmoy 4y ago

Microsoft had marked an email from a professor from an vt.edu domain email address as spam causing me to miss an interview for a PhD funding.

codegeek 4y ago
Microsoft is especially notorious for flagging legit emails as spam if they are not from one of the regular providers.
- cube00 4y ago
  Flagging if you're lucky, they outright 550 refused my mail until I joined their sender program and applied to have my domain unblocked. Then they proceeded to gaslight me claiming my mail was never blocked even after I forwarded their own error messages and IDs back to them.
  - codegeek 4y ago
    yea unfortunately I have seen those as well. It is ridiculous at times.
  - anonydsfsfs 4y ago
    You got a 550? Lucky! When I worked at a non-profit that hosted our own email server, we had many instances where Microsoft would /dev/null our newsletter e-mails. Their servers would give a 250 indicating acceptance, but the e-mail was nowhere to be found (and yes, we checked the spam folder).
- donmcronald 4y ago
  That's nothing compared to the joy of dealing with legit emails that are flagged as high confidence phishing.
- maestroia 4y ago
  Regular provider == (Microsoft 365 || an Exchange Server)
4y ago

aaron_m04 4y ago

I have this part: > I host my own email server with Vultr on an OpenBSD VM using OpenSMTPD and Dovecot But with outgoing mail being relayed internally to dkimproxy which signs it before being relayed back to OpenSMTPD for delivery to the other email server. I had to set up SPF and DKIM DNS records, and one time I had to request that my IP be removed from the Abusix blacklist. Other than that, it's pretty rare for my emails to be marked as spam. Outlook 365 seems to do it much more often than Gmail though.

zcdziura 4y ago
That's very interesting. I never thought to relay mail internally to dkimproxy. I'll have to give that a shot. I like the idea of hosting the entire solution myself and not relying on any 3rd party solutions, but relaying through SMTP2Go was the only thing that I tried that actually solved the problem. Perhaps this will offer a good solution! Thanks!
- wahern 4y ago
  I also use the dkimproxy package, but there's now a third-party OpenSMTPd module that can sign messages in-line.[1] I've always found dkimproxy setup a little confusing compared to a built-in/in-line solution. I might try to switch to the module during the OpenBSD 7.1 upgrade process. [1] I think this is the one I had I mind, though I didn't realize it was already in ports: https://cvsweb.openbsd.org/ports/mail/opensmtpd-filters/dkimsign/ https://cvsweb.openbsd.org/ports/mail/opensmtpd-filters/dkim...
  - throw0101a 4y ago
    If you run a mailing list you generally have to worry about ARC (re-signing 'chain of custody') in addition to DKIM: * https://en.wikipedia.org/wiki/Authenticated_Received_Chain https://en.wikipedia.org/wiki/Authenticated_Received_Chain I've found ARC to fiddle some to get going than ARC.

plainnoodles 4y ago

> There are all kinds of invisible forces that you abutt that can be difficult to figure out This was my main experience, and all I did was try to set up the ability to simply send emails to myself (gmail) (and no-one else). Things like: this script crashed, or btrfs scrub finished + scrub results, and similar. The first thing I tried was just setting up a VM with postfix running on it locally with my residential ISP. I don't even remember what the error was for this scenario, but it was just totally dead in the water. Absolutely zero mail delivery. I think I eventually figured out it's because google defers to spamhaus, and spamhaus says residential IPs = hard no. That next thing I tried, and what I ended up doing, was writing a docker container that just runs an SSH port forward to jump from my local network to a digitalocean host, which is where another docker container runs postfix. I had done this bit once before, and I tried to just set up DKIM (since DKIM was, to my reading, basically bulletproof - why bother with SPF when you have real cryptographic identity assurance?). This led to weird error messages from google about my IP having a super low reputation. This was something I'd been worried about so I spent a bit of time trying to cycle my IP. But I eventually figured out it was just a bad error message and setting up SPF suddenly made my emails start delivering. My main ongoing issue is that I had to add all my sending addresses (things my internalhostnamehere@myrealdomain.com) to my contacts in gmail, otherwise there was like a 50% chance they'd just go to spam. I've been running this setup for about a year and it's still a coin toss whether emails will come through fine, or if they'll say "this would've gone to spam but it's in your contacts". When that happens, I check the DKIM and SPF status in "original message" in gmail, and gmail itself says they both passed. Absurd tbh. For my "not self-hosted but better than letting google own my digital identity" solution, since I use apple icloud+ or whatever it's called, I set up the SPF stuff to let me send+receive email from my custom domain, so while icloud could still scan my mail, at least if I get banned, I still own the actual domain and could move somewhere else.

miohtama 4y ago
Even if one setups everything by the book (SPF, DKIM, DNS.) etc. No one at @outlook.com will receive email, based on my experience. Thus, it does not work well if email is important for business-to-business use. Outlook and Gmail are basically having opaque rules who can receive email and there is no process to get “whitelisted” on these receivers.

leros 4y ago

This is the answer. Blocked emails happen for random reasons and fixing them is a black art that involves talking to ISPs and stuff. It's really too much for an average person to handle. At work we've had issues with email delivery due to things like outdated IP block lists at some random ISP four hops away, only impacting deliverability when mail gets routed through that part of the web.

paulmd 4y ago
I have an email address with "spam" in the name (this is through gmail) and lately I've had all kinds of problems with emails to it disappearing - I've had to call several places and have them change my email because I can't log in and the reset emails don't ever show up... but changing to myfirst.mylast works fine. I've run into this with both Sam's Club and Speedway Rewards. Only thing I can think of is that some outbound mail service they're using is dropping them, or some relay in the middle is dropping them... I can see where the word "spam" would be a keyword you might use, but I've had this email address for 15 years now and it's only been a problem in the last few years.

post-it 4y ago

Prediction: Any distributed social media (like Mastodon) that gains mainstream popularity will share the same fate. Sure, you'll be able to host your own Mastodon instance, but 99% of people will be on the top 10 hosts and they won't peer with you. I think the only way to make distributed social media practical is to have an extremely inexpensive turnkey self-hosting solution for the average person. A Chromecast-like device that they plug into their TV that backs up all their photos, plays music, and also hosts a Mastodon instance. Some kind of very friendly backup solution where you make an "emergency contacts" list, and the device encrypts all of your data and stores it on your emergency contacts' devices as a backup, and vice-versa.

mypalmike 4y ago
On Mastodon, I believe it's currently somewhat backwards from this. The largest instances are filled with Japanese anime porn, and the smaller instances end up blacklisting them.
- rootusrootus 4y ago
  Anecdotally, this has happened every time I've set up any kind of social media instance / discussion forum / BBS (back in the day!) / whatever. It immediately gets consumed by people who use it to host porn, and then all the intended users leave.
  - derefr 4y ago
    Have you considered creating a discussion platform where people can't post images, URLs / things that would be URLs if you added a URL scheme to them, ASCII-armored baseN-encoded anything, etc? For 99% of the discussion you want on the platform, text is all you need. For spammers and people who want to host porn, text alone is useless.
    - sodality2 4y ago
      Plenty of spammers rely on text, though. A good chunk of my spam folder is text only.

deleted 4y ago

[deleted]

tzs 4y ago

> And even if you set everything up properly and host your email with a responsible host, Microsoft will still mark your mail as spam. I did some experiments back when I ran my own mail. Sending from my mail server to my Microsoft account it not only marked everything as spam, it continued marking everything as spam after I marked a bunch of them as not spam. After that, I tried also answering several of them and composing several new mails to send to my non-Microsoft email to see if Microsoft's spam system was smart enough to figure out that if I'm actively corresponding with someone their incoming mail should not be marked as spam. It was not smart enough. Then I tried whitelisting. Nope, still spam.

wpietri 4y ago

Yup. Spam is the root problem. With an enormous amount of complexity between that and the mail admin's day to day experience. I hosted my own mail for more than 20 years. A couple years back I just got tired of trying to solve deliverability puzzles, plus the fears that deliverability issues generate. (E.g., "Did that potential employer get my email about the job?") Especially since some of the puzzles are not solvable, like why GMail does what it does. I even had friends at Google, and I still couldn't find out why GMail occasionally didn't like my server. And arguably, that's the right choice for them, as the more spammers know about how they work, the worse it is for Google staff and GMail users. For me, switching to Fastmail hosting was a big win. It's not like I'm out of technical challenges to solve, but I get to apply that to things where the upside is greater than, "The thing everybody expects to work still works."

clairity 4y ago
the spam problem advantages google, as your own story illustrates, so it's unlikely they'd really want to help solve deliverability/spam issues systemicly. making personal email hosting more difficult means they have a chance to capture your email data streams via gmail. whether you switch there or not, it creates a pressure for most to aggregate on gmail, which means they can see most email exchanges.
- wpietri 4y ago
  For sure. Good spam filtering was one big reason for people to switch to GMail. And a lot of people who gave up hosting their own email have switched to Gmail as well. I'm sure this doesn't rise to the level of conspiracy, but there's little incentive for them to fix the broader problems.
azinman2 4y ago
My issue with fast mail et all is storage is so unnecessarily expensive. I have many gigs of email that I don’t want to lose, but I also don’t want to pay many tens of dollars/month to host it.
- 4y ago

elorant 4y ago

Any chance you'll provide a detailed write-up of your experience with tips and whatnot?

inetknght 4y ago

> The problem is that spam was/is so bad that extreme measures were taken to curb it. The problem with spam is that there's no real legal recourse for spam. If it's in your own country then maybe. But outside of your country? Well the easiest thing to do is to IP block and the next best thing to do (when IP block isn't an option) is to use some sort of "smart detection" to put spam into a special box labeled "spam". There's no deterrence and literally no criminal prosecution for spam.

jjav 4y ago

Yes to the OP, you most definitely can host your own email fully. Many of us do it. If you have any interest in the topic, either due to the fun of managing the servers and learning something along the way or due to the moral high ground of supporting decentralization above proprietary walled gardens, do it! Ignore the naysayers, if you're interested you can do it. Will some emails very occasionally end up in the spam folder of a recipient? I mean, yes, but that is true of everything. You can end up in spam folder sending from Microsoft Office mail to gmail or vice versa. Heck, every now and then an email from my manager will end up in my spam folder in gmail even though he's emailing me from gmail to gmail, both of us in the same corporate gsuite account! So on average, once you set everything up correctly, your deliverability will be as good as gmail to gmail, which is to say not 100% perfect but no worse than any other solution. And you'll be in control of your email infrastructure and address. No longer will google/microsoft/apple/yahoo be able to cut you off all your accounts on the whim an AI gone bad. The parent post mentions a useful safety valve to know about if you're worried about deliverability and want to take baby steps to get there. You can always, either selectively or wholesale, use a commercial relay for outbound mail from your email server. Some have free tiers that are plenty for personal/family use. Personally I don't use any third party relay, I deliver to everywhere from my own infrastructure. No issues.

leephillips 4y ago
Me too, and this is my experience as well. In the rare event that I find out that someone isn’t getting my emails, I tell them that they should complain to their provider or use a different one. I’m no longer willing to jump through hoops so that hotmail delivers my email.

throw10920 4y ago

> The problem is that spam was/is so bad that extreme measures were taken to curb it. Man, and there's such an easy solution, too - just use Hashcash[1] (invented in 1997) and 90%+ of spam disappears overnight (if not more, depending on how high you set the difficulty). Well, ok, "easy" in the sense that We Have An Algorithm For This - it'd still be hard to get email clients/servers to agree on a protocol... [1] https://en.wikipedia.org/wiki/Hashcash https://en.wikipedia.org/wiki/Hashcash

throw0101a 4y ago
If there's a compromised machine it will be the victims paying the cost in energy bills for spammer's nefariously installed malware to send garbage.
- throw10920 4y ago
  If there's a compromised machine, the scammer can drain the victim's bank accounts and cost them far more than an electricity bill, and/or mine cryptocurrency directly. Regardless, their spam-sending rate will still be significantly decreased. So, this argument is completely invalid.
  - throw0101a 4y ago
    > If there's a compromised machine, the scammer can drain the victim's bank accounts […] Not if the machine is a server and was gotten into via (e.g.) a bug in a web app. I don't know about you, but I don't keep my bank account information on the LAMP systems I sysadmin.
    - throw10920 4y ago
      That part of the post was specifically about consumer devices. You missed the rest: > and/or mine cryptocurrency directly. Regardless, their spam-sending rate will still be significantly decreased Your argument remains invalid.
dTal 4y ago

derefr 4y ago

Is there such a service that will tell me the reputation of an email domain, i.e. whether mail originating at that domain would be likely to be treated as definitely spam or not? (I don't really care about "no reputation"; I want to know if a domain has known bad reputation.) I feel like, if there was such a service, it would be pretty useful to use it to prevent account registrations on other services, from users whose email addresses have domains with bad reputations. After all, they'd very likely just be registering with the intent of using the service to send or post spam in some way.

TZubiri 4y ago
multirbl.valli.org Contains blacklists on the domain level, also on the ip block and AS level.
- derefr 4y ago
  Possibly interesting... but these are rules about outgoing SMTP servers (MSAs), yes? How much of a relation does the outgoing SMTP server for a domain have to the canonical set of receiving SMTP servers (MTAs) for the domain held in the domain's DNS MX record? These can certainly be one and the same server; but it's not a requirement. So how often are they in practice? Especially for people actively trying to evade these sorts of RBLs?

patrck 4y ago

Also, one should subscribe to the mailop mailing list, which serves as a Distant Early Warning line for email deliverability issues (ie. like NANOG for netops issues). https://www.mailop.org/best-practices/ https://www.mailop.org/best-practices/

raxxorraxor 4y ago

Spam got significantly worse but this is also an chance to curb the federalization of mail by large companies. Of course they would like you to use a Microsoft or Google account to send mail.

tinroofrusted 4y ago

zddziura, I want to say a big thank you, thank you, thank you for pointing me to SMTP2Go. I have been trying to get my DMARC and DKIM email woes solved for months, but couldn't get it figured out. When I read your post I signed up with SMTP2Go at the free tier and I had a 100% Mailgenius score in less than an hour after I set it up. So awesome! No more big yellow warning boxes in Gmail when receiving mail from my own domain! Yea!!!