8 ms·

Is the implication that this is an intentional practice to drive out small players?

jandrese 6y ago
I don't think it is an attempt to drive out small players, it's a natural consequnce of domains being almost free to stand up. They just don't provide any level of assurance that the owner isn't a spammer. So the blackhole systems treat them as "probably spammer" and fire up a block at the first whiff of malfeasance. It's harder to sign up for a Gmail account than it is to register a domain and get some hosting. And Google has their own protections against sending large amounts of mail built-in. The system is as it is now because spammers will abuse it otherwise.
- throwaway2048 6y ago
  gmail is effectively block by default these days, unless Google has some reason to not block your mail, it will be spamboxed.
  - bachmeier 6y ago
    True. It's a mistake to do a job search using gmail. Interview invitations and job offers are going to go straight to spam.
    - alyandon 6y ago
      Sending email to Google is always a crap shoot and I still find to this day that sometimes when I send email contains logs to my own account using my own account credentials it'll end up being classified as spam. Right now, Google is refusing to deliver email from logwatch - giving me the "Message rejected. See https://support.google.com/mail/answer/69585 https://support.google.com/mail/answer/69585 for more information." so those are going to /dev/null with no recourse to fix it because Google literally doesn't like the content of the email. The fact that I can't hit a button and tell Google that yes indeed I'm not trying to spam myself is ridiculous.
  - vbezhenar 6y ago
    That was the case for me few years ago. But I decided to set up mailserver once more recently and Gmail to my surprise accepted my mails. I configured everything mentioned in article. That said, my outcoming mail traffic is almost non-existent outside of few test emails, LoL.
  - hedora 6y ago
    I have an old, but frequently used email address that got force-moved from a pass through relay to gmail. After setting gmail to forward everything, I noticed it was sending 33% of my legitimate email to spam. They don’t let you disable the spam filter, but you can set up an escoteric filter that prevents it from actually putting stuff in spam. I honestly don’t understand how people cope with gmail. Fastmail is cheap and orders of magnitude better.
    - mulmen 6y ago
      I created my gmail account in 2004 I think. Things were different back then. Now there's just 16 years of momentum preventing me from switching. I like the idea of hosting my own email but even if I get around to doing that I will still need a backup. Fastmail is probably where I will go.
    - Sami_Lehtinen 6y ago
      With Outlook you can't unfortunately do that. Create rules, which would un-spam mail. Because filters aren't applied to spam emails at all.
- dorfsmay 6y ago
  I disagree. The big email players are accepting mail from universities and large companies with misconfigured email servers. If you use the same rules that Google and Microsoft use to refuse emails, you will be bouncing emails from valid domains. If you don't apply them, you end up accepting spam. The big players are able to deal with the spam using specialised teams and I suspect advanced algos including machine learning. IF the big players applied their famous rules to everybody equally, everybody, including universities, big companies etc... Would quickly configure thier mail servers properly, which WOULD reduce spam, possibly eliminate it. Being able to keep inboxes fairly free of spam in a world full of spammers is what distinguis big email providers and enable them to sell their services.
  - peterwwillis 6y ago
    > The big players are able to deal with the spam using specialised teams and I suspect advanced algos including machine learning. I remember when I imagined big companies might use their considerable resources to put together something like that. Then I found out it's just a handful of random engineers cobbling together crap and hiding it behind a fancy domain name. It is probable that Office365's entire spam filtering system is a very large and ugly spreadsheet that gets passed around between a few teams using a ticketing system.
    - vitaemec 6y ago
      I used to work on the O365 antispam team, and I'm not sure how much I'm allowed to say in public, but I'm pretty sure I can publicly say "a very large and ugly spreadsheet that gets passed around between a few teams using a ticketing system" is not accurate. Relevant wikipedia page: https://en.wikipedia.org/wiki/Exchange_Online_Protection https://en.wikipedia.org/wiki/Exchange_Online_Protection Originally, the product started out as an acquisition, and there's still some legacy code that references the old Forefront name. There's about 50 people on the team. DKIM/DMARC/ARC is worked on by two engineers.
csunbird 6y ago
No, I think that there are just too many spammers, so big players have no choice except to make it hard to own your mail server until your server gains positive reputation.
- nbandv 6y ago
  I don't think there are many spammers compared to, say, 2005. In 2005 you would get 100 spams to an unfiltered, widely published address, these days it is more like 2-5. The continued lockdowns and protocol changes of gmail and yahoo are a sign of being overstaffed, "feature" oriented and yes, the attempt to shut down competition and the free exchange of mail.
  - csunbird 6y ago
    I do not think so, I think the spammers are still out there, armed with cheap cloud email providers, but the "block by default" is putting a nice dent on them.
  - user5994461 6y ago
    You're lucky if you only get 2 spams a day. My parents are getting more like 20-50 a day on the email provided by a national ISP. It's getting impossible to read emails, too many. If it were not for them being used to their email address and me being lazy, I would move them off to gmail.
- feanaro 6y ago
  This is not true. They're not even making an effort and are quite probably actively malicious. There is nothing you can do to build up your reputation if you're a small email server. There are no introspection tools to give you a hint on what's wrong and no one to contact. Besides, what are those fancy ML antispam algorithms for if the only cure for spam is "reputation"? It's clearly an undefined term meant to be exclusionary. A person from Gmail even posted on HN a while ago, stating they'll look into this and do better. That was about a year ago and the situation is exactly the same or worse.
EvanAnderson 6y ago
I don't think it's done intentionally, but I also don't think the big players go out of their way not to harm the small ones. All the big email players would love to take your money to "outsource" your email.
g-garron 6y ago
I really think so. They are forcing everybody into their services.
- Proven 6y ago
  How exactly do they "force" you?
kilburn 6y ago
It is, at least if you ask me. People here are speaking about how gmail will "effectively spambox your mails by default". That's not been my experience (from setting up multiple small customers). Anyway, at least I've never heard of gmail just eating your e-mails. They either get rejected or accepted and put somewhere (maybe the spam folder, but at least somewhere). Office365 / hosted Exchange / Outlook Protection or whatever is it called these days... they should be routed to /dev/null by everyone. They just won't track your reputation unless you send them more than 100 mails per day. Does this sound bad? There's more: if they don't have reputation info from you (because they refuse to track it due to the low volume), your mails will go to spam inboxes even when their filters indicate that the message is not spam. And there's more! Don't dare to ever get a bad reputation (i.e.: a user managing to get hacked and their account used to send a couple hundred spam e-mails before triggering your countermeasures). If this happens you are 100% fucked. Now they will DROP your e-mails. Hear, hear: their servers will accept your mails and just DELETE them. No spam folder, nothing. You will try every possible thing: set up everything for their feedback loop, sign up in their "Smart Network Data Services" to track your reputation (it will be empty except for that day)... and finally contacting them at their sender support. Do you want to know what they will reply? That you should be patient and let your reputation build up over time. What a joke! How on earth can your reputation improve when users cannot mark your mails as "not spam" because they (outlook, not the users) are simply DELETING them without a trace? Oh, there's a way out of all this though: obtain a "Return Path Certification" [1]. That is, pay them an absurd amount of money and your mails are guaranteed to get to the users' inbox unless you are clearly spamming (all of the above assumed you are NOT). Up to this final point you could think they just do their best and all that I've explained is collateral damage. That last "pay and we let you off the hook" is what clearly signals to me that this is an elaborate scheme to get small players to either pay them anyway or just give up and use a big-provider service. [1] https://sendersupport.olc.protection.outlook.com/pm/services.aspx https://sendersupport.olc.protection.outlook.com/pm/services...
- jeffbee 6y ago
  > a user managing to get hacked and their account used to send a couple hundred spam e-mails For this reason one of the necessities of survival as an email sender is to classify your own outbound messages, to see if recipients might think they are spammy. If they just look spammy but you still want to send them, you need sacrificial IP addresses on separate netblocks that are dedicated to spam, so the reputation of your main IPs isn't polluted. If there's any possibility that any of your users could get their account hijacked, or there could be malware on any device permitted to send messages, you need outbound classification.

I don't think it is an attempt to drive out small players, it's a natural consequnce of domains being almost free to stand up. They just don't provide any level of assurance that the owner isn't a spammer. So the blackhole systems treat them as "probably spammer" and fire up a block at the first whiff of malfeasance. It's harder to sign up for a Gmail account than it is to register a domain and get some hosting. And Google has their own protections against sending large amounts of mail built-in. The system is as it is now because spammers will abuse it otherwise.

throwaway2048 6y ago
gmail is effectively block by default these days, unless Google has some reason to not block your mail, it will be spamboxed.
- bachmeier 6y ago
  True. It's a mistake to do a job search using gmail. Interview invitations and job offers are going to go straight to spam.
  - alyandon 6y ago
    Sending email to Google is always a crap shoot and I still find to this day that sometimes when I send email contains logs to my own account using my own account credentials it'll end up being classified as spam. Right now, Google is refusing to deliver email from logwatch - giving me the "Message rejected. See https://support.google.com/mail/answer/69585 https://support.google.com/mail/answer/69585 for more information." so those are going to /dev/null with no recourse to fix it because Google literally doesn't like the content of the email. The fact that I can't hit a button and tell Google that yes indeed I'm not trying to spam myself is ridiculous.
- vbezhenar 6y ago
  That was the case for me few years ago. But I decided to set up mailserver once more recently and Gmail to my surprise accepted my mails. I configured everything mentioned in article. That said, my outcoming mail traffic is almost non-existent outside of few test emails, LoL.
- hedora 6y ago
  I have an old, but frequently used email address that got force-moved from a pass through relay to gmail. After setting gmail to forward everything, I noticed it was sending 33% of my legitimate email to spam. They don’t let you disable the spam filter, but you can set up an escoteric filter that prevents it from actually putting stuff in spam. I honestly don’t understand how people cope with gmail. Fastmail is cheap and orders of magnitude better.
  - mulmen 6y ago
    I created my gmail account in 2004 I think. Things were different back then. Now there's just 16 years of momentum preventing me from switching. I like the idea of hosting my own email but even if I get around to doing that I will still need a backup. Fastmail is probably where I will go.
  - Sami_Lehtinen 6y ago
    With Outlook you can't unfortunately do that. Create rules, which would un-spam mail. Because filters aren't applied to spam emails at all.
dorfsmay 6y ago
I disagree. The big email players are accepting mail from universities and large companies with misconfigured email servers. If you use the same rules that Google and Microsoft use to refuse emails, you will be bouncing emails from valid domains. If you don't apply them, you end up accepting spam. The big players are able to deal with the spam using specialised teams and I suspect advanced algos including machine learning. IF the big players applied their famous rules to everybody equally, everybody, including universities, big companies etc... Would quickly configure thier mail servers properly, which WOULD reduce spam, possibly eliminate it. Being able to keep inboxes fairly free of spam in a world full of spammers is what distinguis big email providers and enable them to sell their services.
- peterwwillis 6y ago
  > The big players are able to deal with the spam using specialised teams and I suspect advanced algos including machine learning. I remember when I imagined big companies might use their considerable resources to put together something like that. Then I found out it's just a handful of random engineers cobbling together crap and hiding it behind a fancy domain name. It is probable that Office365's entire spam filtering system is a very large and ugly spreadsheet that gets passed around between a few teams using a ticketing system.
  - vitaemec 6y ago
    I used to work on the O365 antispam team, and I'm not sure how much I'm allowed to say in public, but I'm pretty sure I can publicly say "a very large and ugly spreadsheet that gets passed around between a few teams using a ticketing system" is not accurate. Relevant wikipedia page: https://en.wikipedia.org/wiki/Exchange_Online_Protection https://en.wikipedia.org/wiki/Exchange_Online_Protection Originally, the product started out as an acquisition, and there's still some legacy code that references the old Forefront name. There's about 50 people on the team. DKIM/DMARC/ARC is worked on by two engineers.

csunbird 6y ago

No, I think that there are just too many spammers, so big players have no choice except to make it hard to own your mail server until your server gains positive reputation.

nbandv 6y ago
I don't think there are many spammers compared to, say, 2005. In 2005 you would get 100 spams to an unfiltered, widely published address, these days it is more like 2-5. The continued lockdowns and protocol changes of gmail and yahoo are a sign of being overstaffed, "feature" oriented and yes, the attempt to shut down competition and the free exchange of mail.
- csunbird 6y ago
  I do not think so, I think the spammers are still out there, armed with cheap cloud email providers, but the "block by default" is putting a nice dent on them.
- user5994461 6y ago
  You're lucky if you only get 2 spams a day. My parents are getting more like 20-50 a day on the email provided by a national ISP. It's getting impossible to read emails, too many. If it were not for them being used to their email address and me being lazy, I would move them off to gmail.
feanaro 6y ago
This is not true. They're not even making an effort and are quite probably actively malicious. There is nothing you can do to build up your reputation if you're a small email server. There are no introspection tools to give you a hint on what's wrong and no one to contact. Besides, what are those fancy ML antispam algorithms for if the only cure for spam is "reputation"? It's clearly an undefined term meant to be exclusionary. A person from Gmail even posted on HN a while ago, stating they'll look into this and do better. That was about a year ago and the situation is exactly the same or worse.

EvanAnderson 6y ago

I don't think it's done intentionally, but I also don't think the big players go out of their way not to harm the small ones. All the big email players would love to take your money to "outsource" your email.

g-garron 6y ago

I really think so. They are forcing everybody into their services.

Proven 6y ago
How exactly do they "force" you?

kilburn 6y ago

It is, at least if you ask me. People here are speaking about how gmail will "effectively spambox your mails by default". That's not been my experience (from setting up multiple small customers). Anyway, at least I've never heard of gmail just eating your e-mails. They either get rejected or accepted and put somewhere (maybe the spam folder, but at least somewhere). Office365 / hosted Exchange / Outlook Protection or whatever is it called these days... they should be routed to /dev/null by everyone. They just won't track your reputation unless you send them more than 100 mails per day. Does this sound bad? There's more: if they don't have reputation info from you (because they refuse to track it due to the low volume), your mails will go to spam inboxes even when their filters indicate that the message is not spam. And there's more! Don't dare to ever get a bad reputation (i.e.: a user managing to get hacked and their account used to send a couple hundred spam e-mails before triggering your countermeasures). If this happens you are 100% fucked. Now they will DROP your e-mails. Hear, hear: their servers will accept your mails and just DELETE them. No spam folder, nothing. You will try every possible thing: set up everything for their feedback loop, sign up in their "Smart Network Data Services" to track your reputation (it will be empty except for that day)... and finally contacting them at their sender support. Do you want to know what they will reply? That you should be patient and let your reputation build up over time. What a joke! How on earth can your reputation improve when users cannot mark your mails as "not spam" because they (outlook, not the users) are simply DELETING them without a trace? Oh, there's a way out of all this though: obtain a "Return Path Certification" [1]. That is, pay them an absurd amount of money and your mails are guaranteed to get to the users' inbox unless you are clearly spamming (all of the above assumed you are NOT). Up to this final point you could think they just do their best and all that I've explained is collateral damage. That last "pay and we let you off the hook" is what clearly signals to me that this is an elaborate scheme to get small players to either pay them anyway or just give up and use a big-provider service. [1] https://sendersupport.olc.protection.outlook.com/pm/services.aspx https://sendersupport.olc.protection.outlook.com/pm/services...

jeffbee 6y ago
> a user managing to get hacked and their account used to send a couple hundred spam e-mails For this reason one of the necessities of survival as an email sender is to classify your own outbound messages, to see if recipients might think they are spammy. If they just look spammy but you still want to send them, you need sacrificial IP addresses on separate netblocks that are dedicated to spam, so the reputation of your main IPs isn't polluted. If there's any possibility that any of your users could get their account hijacked, or there could be malware on any device permitted to send messages, you need outbound classification.