7 ms·

I wouldn't be surprised if that's intentional. There's an explicit hesitance on the part of mail providers to accept v6 mail, since they use IP addresses as a r

I wouldn't be surprised if that's intentional. There's an explicit hesitance on the part of mail providers to accept v6 mail, since they use IP addresses as a reputation mechanism. IPs that originate spam mail get summarily executed, and getting new IPs that have a high antispam reputation is actually quite expensive. In other words, it's a Sybil-resistance mechanism, called Proof-of-IPv4. It works specifically because v4 addresses are scarce. v6 addresses are not nearly as such. Everything that makes IPv6 great for the Internet at large makes it terrible for mail providers. For example, because the original v6 design wanted to eat lower link layers, it reserves half the v6 address for an embedded MAC64. This never really panned out, but it's terrible for security, so every v6-capable OS nowadays will rotate addresses every few hours. The average machine will have hundreds of addresses. How do you assign a usable notion of per-IP reputation to that? You could use v6 subnets for reputation, but there's still 64 subnet bits - enough to stick an entire IPv4 subnetwork inside of each IPv4 address. Some ISPs actually will assign a /64 per customer (because Comcast needs something to sell to Business customers), while others assign /56s or /48s. So there isn't even one granularity of subnetting that you can use for reputation tracking on v6. Meanwhile, v4 pricing is getting worse and worse, which is great for mail providers. They don't necessarily need to turn a profit on incoming mail, but they do need to make it expensive for people who want to send lots of spam.

adevx 5y ago
This could likely be the reason for poor IPv6 support but highlights the importance of shifting (much more) to domain based reputation. If a domains reputation is at risk, you can bet domain holders will be extremely careful not to allow outgoing spam.
- syshum 5y ago
  Or more strict enforcement by the world on SPF, DMARC and DKIM policies The problem of spam is actually solved, the problem is no one setups any of these security parameters correct, large and small companies alike all have bad SPF Records, bad or no DMARC, etc etc etc
  - jtchang 5y ago
    How is that solved then if no one setups any of the security parameters correctly? That sounds like the exact opposite.
    - teknopaul 5y ago
      https "solves" Internet security, your bad of you don't use it.
  - friendzis 5y ago
    Go to any internet-related forum and search history for those keywords. You will find countless stories of seemingly technically people who in the end give up on self hosting and switch to managed mail provider. Because even if you solve those policies perfectly, a personal mail server will have such a low rate of outgoing mail that all the big players will effectively treat it as history-less server and will occasionally route the mail into the black hole. There is no recourse for that. If 99% of contacts you want to send mail to are on google/yahoo/microsoft you have to play by their rules. And those rules are effectively "send mail internally or gtfo".
    - nanidin 5y ago
      I have self hosted personal mail for over a decade. There are occasional hiccups with deliverability to new gmail addresses, but that is it. In those cases, once a recipient marks me as not spam once, there aren’t any more problems. I think maybe once in the last 3 years I ended up in someone’s spam box, total. In fact I just sent to a new gmail address and to a university I have never contacted before this week and both were delivered without issue. Setting up DKIM/SPF/etc isn’t that hard and it’s fairly easy to verify with existing tools FYI.
      - generalizations 5y ago
        It would be amazing if those of you who have successfully self-hosted would get together and make a comprehensive write-up of how to self-host without getting blacklisted. I frequently see comments on both sides of this ("I can't send anything!" vs. "You just have to do it right") and it seems like if there could be some resource (that cuts down the complexity as far as is reasonable) on how to do this from the ground up, there could be much more wide-spread adoption of self-hosting. Personally, I'm hesitant because I don't know if the end of all my effort will be constant blacklisting. If I could be confident that if I do it right I won't get blacklisted, I probably would.
- throw0101a 5y ago
  > If a domains reputation is at risk, you can bet domain holders will be extremely careful not to allow outgoing spam. Generating domains is fairly cheap though. lsjfdlakj.com There, I just generated a new one with a clean reputation. Just spend US$ 10 to register it and off we go.
  - wrycoder 5y ago
    It has no reputation. That's different from a 'clean' reputation, which takes history to establish.
  - adevx 5y ago
    You often have to build a domain reputation first. Certainly for Microsoft hosted email. I for instance show users with a Microsoft email a plain mailto:support@domain.tld link on my contact/support form. This way the first email is from them to me which helps building reputation and minimizes the chances of my response going straight into the spam box or worse, silently dropped. Regular users can fill in a proper form and submit it from the support page.
    - amichal 5y ago
      I like it. I've always wanted to promote mailto: links over silly contact us forms (and all the hoops you have to jump through to keep them functional and not abused) but never had a really good argument for non-techy folks and lots of pushback that mailto: is "not standard" and "does not work for some people" with very little evidence. This is a nice story for the 'pros' column.
    - teknopaul 5y ago
      I get more spam from Microsoft and other tech giants than anyone else. It's the companies whom you rely on for email that are the worst abuser e.g. airlines need to inform you about delays and abuse this trust with holiday adverts incessantly. Any company that claims to require your email for two factor auth should be given automatically generated fines for every email they ever send that is not auth related. That would shake up Oracle sales dept. :)
- rinron 5y ago
  Spammers and scammers already use domains as a disposable commodity creating them or using hacked ones for single campaigns and moving on. Part of filtering based on IPv4 is not only scarcity but accountability. When the owner of the netblock reassigns the ip and its already blacklisted it can create a problem for them and incentivize them to police their own network. Domains are also worse in that its easier to use fake information and be untraceable. its also understandably easier to get a response legal or otherwise from a co-location or isp than a domain registrar. Maybe ipv4 will always be preferred for email just because its more difficult/expensive and therefore less appealing for temporary malicious use.
- kmeisthax 5y ago
  Domains are less scarce than addresses. By design you can create as many subdomains as you like. (e.g. `abc.spam.com` is too low-rep? Now let's try `def.spam.com`...) You might imagine negative reputation to travel up to parent domains, but that causes problems with public suffixes and TLDs. (e.g. is `microsoft.com` a bad domain because it's got the same TLD as `spam.com`?) The whole point of using a scarce identifier is to allow for a "neutral" reputation for new identifiers. If identifiers are less scarce, then known-bad actors can get free reputation (from bad to neutral) by just starting over with a new one. Which means that you have to distrust neutral reputation more. Without some level of scarcity of identification, introductions don't really work, because I have no idea if the new host I'm being talked to from today is just the one I banned yesterday wearing a different mask. This ultimately implies e-mail moving to some kind of federated whitelist system rather than the current system of federated blacklists.
  - Ruphin 5y ago
    Subdomains are not as much a problem as you would think. There is a resource ( https://publicsuffix.org/ https://publicsuffix.org/ ) that lists all public suffix domains. All direct subdomains of these are in full control of all their own subdomains, and thus can share the same spam reputation. e.g. when .com is on the list, and .somesite.com is not on the list, mail@somesite.com is from the same entity as mail@subdomain.somesite.com
    - kelnos 5y ago
      Didn't publicsuffix effectively get DoSed by one of Apple's new requirements, causing a ton of people to apply to have their suffixes added to the list? From what I gathered from that, publicsuffix is a poorly-funded semi-volunteer org that shouldn't be relied upon for anything critical.
      - cormacrelf 5y ago
        If this is how MS/Google want to do email anti-spam, they should fund the public suffix list. Same with Apple for their App Store and WebKit uses. (Btw I’m pretty sure almost everyone is already using domain-based spam scoring.)
- tikiman163 5y ago
  Filtering based on domain reputation has the same problem as trying to filter based IPv6 address reputation. They can easily change thier domain name at any time, and most spam operations do it every 15 minutes or so. This also has a secondary problem for legitimate domain buyers. If the domain name they buy was previously used for spam that reputation will affect thier business for quite a while. There's actually a market where people buy domains with bad reputations, setup small legitimate businesses and get the reputation cleaned up, then sell the site domain and business for a substantial profit because a site with a good reputation history and established line of business will show up higher on internet searches.
blibble 5y ago
I'm surprised there's not some sort of database which records the size of subnets allocated to end-users would be very useful (business opportunity here guys!)
- mfrye0 5y ago
  I've been working on this and have built that database, though we only expose at the IP level: https://bigpicture.io/docs/api/#ip-api https://bigpicture.io/docs/api/#ip-api. What did you have in mind as far as a use case?
  - blibble 5y ago
    given abuse coming from a given IPv6 address: which subnet do I need to block to stop the user behind that address (for fraud detection it switches from block to identify) for IPv4 this is generally the /32 (the single IPv4 address) for IPv6 it's probably a /64, but may be a /56 or even a /48, and on some crappy providers even a /128 if the subnet is smaller than you think it is you risk banning an entire ISP (or country), whereas if if it's too large the abuse continues it's quite a complicated problem as by design you can have subletting (subnetting!) within a block, e.g. a VPS provider gets a /48 from its ISP, and then they sublets out /64s to their customers (while not necessarily giving them all their own RIPE/ARIN records)
    - mfrye0 5y ago
      Got it. Yeah, it's definitely tricky. The other aspect is that a decent chunk of the IPv4 space at least is fairly dynamic. We've seen some blocks change owners every few weeks.
    - 2Gkashmiri 5y ago
      can i ask a question? is it possible for people to "own" ipv4 addresses? like we can own domain names? something like /29 Subnet or /28? if i spent like a hundred bucks or something, i dont know... just asking. how would that work, does that "bring your own ip" that vps providers talk about mean this? i
cm2187 5y ago
You could have a reputation based on /64 and to extend the subnet when you see a large number of spam coming from the same /56 or /48.
- sneak 5y ago
  This is a perfectly reasonable approach that mirrors that of the current ipv4 reputation scheme. Treating individual v6 addresses like individual v4 addresses is silly and nobody serious will take that approach.
- deleted 5y ago
  [deleted]
- tgragnato 5y ago
  Classifying IP sets is a fantastic idea, I’ve seen mail bounce for the ASN. That parameter is unchanged between IPv4 and IPv6. Certainly, you can do it only when the provider is a classic spam heaven.
dathinab 5y ago
> do need to make it expensive for people who want to send lots of spam. You can use cloud providers, sure small ones do get blacklisted (which happens to also benefit Microsoft as they also are a cloud provider) but they can't really blacklist Googles or Amazons Cloud.
- kmeisthax 5y ago
  Google is not a good place to send spam. They'll delete your account and ban the cell number you used to SMS verify.
GoblinSlayer 5y ago
Can't the reputation mechanism rely on DKIM for identification?
teknopaul 5y ago
seems trivially resolvable for respectable email providers, just don't change your ipv6 source address.
- drnex 5y ago
  the problem is that spammers can change their ipv6 addresses at less cost than ipv4
MrStonedOne 5y ago
>Some ISPs actually will assign a /64 per customer (because Comcast needs something to sell to Business customers), while others assign /56s or /48s. So there isn't even one granularity of subnetting that you can use for reputation tracking on v6. This is what is hindering adoption everywhere to be honest. All of my forums, wikis, and game servers reject ipv6 purely for the same reason.
- namibj 5y ago
  How is blanket-blocking better than assuming the worst and attaching reputation to /48's?
binkHN 5y ago
> Some ISPs actually will assign a /64 per customer (because Comcast needs something to sell to Business customers) FWIW, I actually have residential service from Comcast and they'll assign you a /60 if you request it. I then use /64s for my subnets/VLANs.
lugged 5y ago
> How do you assign a usable notion of per-IP reputation to that? You don't and that's the point. Stop bloody tracking me. What should you as a mail provider do with this new information? Oh I dunno, innovate?

adevx 5y ago

This could likely be the reason for poor IPv6 support but highlights the importance of shifting (much more) to domain based reputation. If a domains reputation is at risk, you can bet domain holders will be extremely careful not to allow outgoing spam.

syshum 5y ago
Or more strict enforcement by the world on SPF, DMARC and DKIM policies The problem of spam is actually solved, the problem is no one setups any of these security parameters correct, large and small companies alike all have bad SPF Records, bad or no DMARC, etc etc etc
- jtchang 5y ago
  How is that solved then if no one setups any of the security parameters correctly? That sounds like the exact opposite.
  - teknopaul 5y ago
    https "solves" Internet security, your bad of you don't use it.
- friendzis 5y ago
  Go to any internet-related forum and search history for those keywords. You will find countless stories of seemingly technically people who in the end give up on self hosting and switch to managed mail provider. Because even if you solve those policies perfectly, a personal mail server will have such a low rate of outgoing mail that all the big players will effectively treat it as history-less server and will occasionally route the mail into the black hole. There is no recourse for that. If 99% of contacts you want to send mail to are on google/yahoo/microsoft you have to play by their rules. And those rules are effectively "send mail internally or gtfo".
  - nanidin 5y ago
    I have self hosted personal mail for over a decade. There are occasional hiccups with deliverability to new gmail addresses, but that is it. In those cases, once a recipient marks me as not spam once, there aren’t any more problems. I think maybe once in the last 3 years I ended up in someone’s spam box, total. In fact I just sent to a new gmail address and to a university I have never contacted before this week and both were delivered without issue. Setting up DKIM/SPF/etc isn’t that hard and it’s fairly easy to verify with existing tools FYI.
    - generalizations 5y ago
      It would be amazing if those of you who have successfully self-hosted would get together and make a comprehensive write-up of how to self-host without getting blacklisted. I frequently see comments on both sides of this ("I can't send anything!" vs. "You just have to do it right") and it seems like if there could be some resource (that cuts down the complexity as far as is reasonable) on how to do this from the ground up, there could be much more wide-spread adoption of self-hosting. Personally, I'm hesitant because I don't know if the end of all my effort will be constant blacklisting. If I could be confident that if I do it right I won't get blacklisted, I probably would.
throw0101a 5y ago
> If a domains reputation is at risk, you can bet domain holders will be extremely careful not to allow outgoing spam. Generating domains is fairly cheap though. lsjfdlakj.com There, I just generated a new one with a clean reputation. Just spend US$ 10 to register it and off we go.
- wrycoder 5y ago
  It has no reputation. That's different from a 'clean' reputation, which takes history to establish.
- adevx 5y ago
  You often have to build a domain reputation first. Certainly for Microsoft hosted email. I for instance show users with a Microsoft email a plain mailto:support@domain.tld link on my contact/support form. This way the first email is from them to me which helps building reputation and minimizes the chances of my response going straight into the spam box or worse, silently dropped. Regular users can fill in a proper form and submit it from the support page.
  - amichal 5y ago
    I like it. I've always wanted to promote mailto: links over silly contact us forms (and all the hoops you have to jump through to keep them functional and not abused) but never had a really good argument for non-techy folks and lots of pushback that mailto: is "not standard" and "does not work for some people" with very little evidence. This is a nice story for the 'pros' column.
  - teknopaul 5y ago
    I get more spam from Microsoft and other tech giants than anyone else. It's the companies whom you rely on for email that are the worst abuser e.g. airlines need to inform you about delays and abuse this trust with holiday adverts incessantly. Any company that claims to require your email for two factor auth should be given automatically generated fines for every email they ever send that is not auth related. That would shake up Oracle sales dept. :)
rinron 5y ago
Spammers and scammers already use domains as a disposable commodity creating them or using hacked ones for single campaigns and moving on. Part of filtering based on IPv4 is not only scarcity but accountability. When the owner of the netblock reassigns the ip and its already blacklisted it can create a problem for them and incentivize them to police their own network. Domains are also worse in that its easier to use fake information and be untraceable. its also understandably easier to get a response legal or otherwise from a co-location or isp than a domain registrar. Maybe ipv4 will always be preferred for email just because its more difficult/expensive and therefore less appealing for temporary malicious use.
kmeisthax 5y ago
Domains are less scarce than addresses. By design you can create as many subdomains as you like. (e.g. `abc.spam.com` is too low-rep? Now let's try `def.spam.com`...) You might imagine negative reputation to travel up to parent domains, but that causes problems with public suffixes and TLDs. (e.g. is `microsoft.com` a bad domain because it's got the same TLD as `spam.com`?) The whole point of using a scarce identifier is to allow for a "neutral" reputation for new identifiers. If identifiers are less scarce, then known-bad actors can get free reputation (from bad to neutral) by just starting over with a new one. Which means that you have to distrust neutral reputation more. Without some level of scarcity of identification, introductions don't really work, because I have no idea if the new host I'm being talked to from today is just the one I banned yesterday wearing a different mask. This ultimately implies e-mail moving to some kind of federated whitelist system rather than the current system of federated blacklists.
- Ruphin 5y ago
  Subdomains are not as much a problem as you would think. There is a resource ( https://publicsuffix.org/ https://publicsuffix.org/ ) that lists all public suffix domains. All direct subdomains of these are in full control of all their own subdomains, and thus can share the same spam reputation. e.g. when .com is on the list, and .somesite.com is not on the list, mail@somesite.com is from the same entity as mail@subdomain.somesite.com
  - kelnos 5y ago
    Didn't publicsuffix effectively get DoSed by one of Apple's new requirements, causing a ton of people to apply to have their suffixes added to the list? From what I gathered from that, publicsuffix is a poorly-funded semi-volunteer org that shouldn't be relied upon for anything critical.
    - cormacrelf 5y ago
      If this is how MS/Google want to do email anti-spam, they should fund the public suffix list. Same with Apple for their App Store and WebKit uses. (Btw I’m pretty sure almost everyone is already using domain-based spam scoring.)
tikiman163 5y ago
Filtering based on domain reputation has the same problem as trying to filter based IPv6 address reputation. They can easily change thier domain name at any time, and most spam operations do it every 15 minutes or so. This also has a secondary problem for legitimate domain buyers. If the domain name they buy was previously used for spam that reputation will affect thier business for quite a while. There's actually a market where people buy domains with bad reputations, setup small legitimate businesses and get the reputation cleaned up, then sell the site domain and business for a substantial profit because a site with a good reputation history and established line of business will show up higher on internet searches.

blibble 5y ago

I'm surprised there's not some sort of database which records the size of subnets allocated to end-users would be very useful (business opportunity here guys!)

mfrye0 5y ago
I've been working on this and have built that database, though we only expose at the IP level: https://bigpicture.io/docs/api/#ip-api https://bigpicture.io/docs/api/#ip-api. What did you have in mind as far as a use case?
- blibble 5y ago
  given abuse coming from a given IPv6 address: which subnet do I need to block to stop the user behind that address (for fraud detection it switches from block to identify) for IPv4 this is generally the /32 (the single IPv4 address) for IPv6 it's probably a /64, but may be a /56 or even a /48, and on some crappy providers even a /128 if the subnet is smaller than you think it is you risk banning an entire ISP (or country), whereas if if it's too large the abuse continues it's quite a complicated problem as by design you can have subletting (subnetting!) within a block, e.g. a VPS provider gets a /48 from its ISP, and then they sublets out /64s to their customers (while not necessarily giving them all their own RIPE/ARIN records)
  - mfrye0 5y ago
    Got it. Yeah, it's definitely tricky. The other aspect is that a decent chunk of the IPv4 space at least is fairly dynamic. We've seen some blocks change owners every few weeks.
  - 2Gkashmiri 5y ago
    can i ask a question? is it possible for people to "own" ipv4 addresses? like we can own domain names? something like /29 Subnet or /28? if i spent like a hundred bucks or something, i dont know... just asking. how would that work, does that "bring your own ip" that vps providers talk about mean this? i

cm2187 5y ago

You could have a reputation based on /64 and to extend the subnet when you see a large number of spam coming from the same /56 or /48.

sneak 5y ago
This is a perfectly reasonable approach that mirrors that of the current ipv4 reputation scheme. Treating individual v6 addresses like individual v4 addresses is silly and nobody serious will take that approach.
deleted 5y ago
[deleted]
tgragnato 5y ago
Classifying IP sets is a fantastic idea, I’ve seen mail bounce for the ASN. That parameter is unchanged between IPv4 and IPv6. Certainly, you can do it only when the provider is a classic spam heaven.

dathinab 5y ago

> do need to make it expensive for people who want to send lots of spam. You can use cloud providers, sure small ones do get blacklisted (which happens to also benefit Microsoft as they also are a cloud provider) but they can't really blacklist Googles or Amazons Cloud.

kmeisthax 5y ago
Google is not a good place to send spam. They'll delete your account and ban the cell number you used to SMS verify.

GoblinSlayer 5y ago

Can't the reputation mechanism rely on DKIM for identification?

teknopaul 5y ago

seems trivially resolvable for respectable email providers, just don't change your ipv6 source address.

drnex 5y ago
the problem is that spammers can change their ipv6 addresses at less cost than ipv4

MrStonedOne 5y ago

>Some ISPs actually will assign a /64 per customer (because Comcast needs something to sell to Business customers), while others assign /56s or /48s. So there isn't even one granularity of subnetting that you can use for reputation tracking on v6. This is what is hindering adoption everywhere to be honest. All of my forums, wikis, and game servers reject ipv6 purely for the same reason.

namibj 5y ago
How is blanket-blocking better than assuming the worst and attaching reputation to /48's?

binkHN 5y ago

> Some ISPs actually will assign a /64 per customer (because Comcast needs something to sell to Business customers) FWIW, I actually have residential service from Comcast and they'll assign you a /60 if you request it. I then use /64s for my subnets/VLANs.

lugged 5y ago

> How do you assign a usable notion of per-IP reputation to that? You don't and that's the point. Stop bloody tracking me. What should you as a mail provider do with this new information? Oh I dunno, innovate?