7 ms·

Email validation keeps being a problem because the rules keep changing. The assumption that .pizza is not a valid TLD was valid before, but is not valid anymore

Email validation keeps being a problem because the rules keep changing. The assumption that .pizza is not a valid TLD was valid before, but is not valid anymore, but not everyone has updated themselves to understand these new rules. It could also be that they see a lot of fraud from people using non-traditional TLDs (basically the gTLDs) so they block those, but serve some generic error that looks like their email is not actually valid. But then email validation has basically always been an issue. The only real way of validating emails is to send an actual email to specified address and ask the user to input something that that email contains, but that adds a lot of work than some simple regex. I also used to use a uncommon gTLD, but then we're not "normal internet users", at this point at least, maybe in the future. Best work-around I came up with is having a traditional TLD that I can use when the system doesn't work with the email I usually use. I think this is the best thing you can do moving forward, because Sony is not the only ones fucking this up, there are plenty of services that don't accept your email if you use something like .pizza, .app or any other common gTLD. Edit: I should clarify. "Rules" is referring to regex rules, for the ones who are engaging in the fruitless endeavour of trying to validate emails with regex, not the "rules" as the specifications for what emails addresses are actually valid. I restate what I said elsewhere: only proper way to validate emails is by sending a email to the address and make the user input some data you sent in said email. Anything else is bound to not work for X% of users.

mro_name 5y ago
> because the rules keep changing the single relevant rule is the same ever since: do an opt in. The spelling of the email address is of zero relevance and must not be nosed at.
- Semaphor 5y ago
  You forget one thing: People make typos. Assuming Sony blocks all LEN(TLD)>3, do you think they’ll lose more customers than they’ll lose from them not getting their validation mail because they typoed their mail? I’d bet that the first group is far smaller than the 2nd. Now instead of blocking it would be nice if they just asked if you are really sure that’s your email, but it’s probably still better than just accepting everything and trying to send a confirmation, both from a business and from a customer-friendliness perspective.
  - mro_name 5y ago
    haven't forgotton. Typos happen not only in the tld. So check and warn if you will. But indeed let it happen and don't gag people. You need the opt-in and have people confirm or repeat in case.
- amerkhalid 5y ago
  Ran into this issue at one of the job. Developers wanted to implement validation based on RFCs. But QA and business refused to believe RFC rules and instead wanted their own rules. Guess what we end up implementing.
  - Freak_NL 5y ago
    What did you end up doing? Anything more than checking if the string contains @ is wrong, and you should be able to find dozens of articles supporting this argument by now.
    - amerkhalid 5y ago
      We checked for dot in addition to @. There was probably also check for special chars which makes sense, iirc. One win we had was, we didn't hardcode any tlds.
  - mro_name 5y ago
    even a RFC doesn't guarantee the existence of a mailbox - if the address e.g. has a typo in the name. And own rules very much sounds like management ever wanted to invent something.
viraptor 5y ago
> The assumption that .pizza is not a valid TLD was valid before, Why do you think so? We didn't have a strict set of TLDs before as far as I know. We just stagnated for a long time. But the list of TLDs has changed many times with countries being created/renamed. If you saw .pizza or .zx, systems should've behaved the same - tried resolving.
- chockchocschoir 5y ago
  > Why do you think so? Because .pizza wasn't a valid TLD before 2014 when it got delegated? Don't know what to tell you, .pizza haven't always existed, it's a relatively new gTLD that got introduced by the new gTLD program by ICANN. But I agree with you in general, trying to validate emails with regex will never work in practice, and has lots of issues. Only real way to validate a address is to actually send email, ask user to input data from said email.
  - roel_v 5y ago
    "trying to validate emails with regex will never work in practice" Of course it does. It just depends on how you define 'validate emails'. Your statement is ambiguous. The Friedl has an entire appendix on a single regex that validates an email address to the format described in RFC 5322, although of course maybe a proper parser would be better, from a technical perspective. However, an email address being in the right format does not ensure that the user that email address is 'associated' with (for whatever definition of 'associated' that is appropriate to the application we're talking about) can/will actually receive emails send to that address. You should do both. Do a first 'pass' validation to check the user didn't make a type. This is more for user convenience, as this can give immediate feedback. And then send an actual email to see if email is deliverable there. The problem here is not that they checked the format. The problem is that they check the wrong way. Although it's not entirely unthinkable that it's sometimes appropriate to add additional constraints to which addresses you want to accept, e.g. banning certain tld's, or even only accepting whitelisted ones. If your fraud stats say that doing that reduces loss, it's not unreasonable to add such a third filter layer.
  - viraptor 5y ago
jeroenhd 5y ago
The rules never changed, the TLD list did. The problem is that people like to put restrictions on which emails are "normal" and which are not. I wouldn't be surprised if websites like these only accept Gmail and Outlook email addresses with their nonsensical restrictions. Blocking an entire TLD because of fraud is insanity. There are plenty of realistic looking domains available for cheap in the dot com TLD, no need to go for dot pizza or dot dev to bypass spam filters.
- dhosek 5y ago
  >Blocking an entire TLD because of fraud is insanity. Wikipedia does this with .xyz links. When I asked about this (I wanted to put a link to finl.xyz on my user page), I was told that this was because of link spamming.
  - Avamander 5y ago
    .xyz gets your spam score increased as well by a lot of mail filters. Price and stricter TLD rules do work as a deterrence to abuse.
    - Brian_K_White 5y ago
      Meanwhile xyz is a perfectly legit tld for sites about 3d printing, cnc machining, 3d modelling, etc. But get stigmatized and prejudged through no fault of their own. It's like name racism.
Aissen 5y ago
The rules never changed. One should only match if the email contains "@" (or better, use browser email inputs), and do the verification by sending an email to it (as you said). Validating TLDs was never a good idea, and still isn't. Yes, you can put UX helpers for spelling mistakes on most common domains (gmail…) (helps the majority), but never make it a hard error. It's that simple.
- hu3 5y ago
  IIRC the RFCs for email address specify mandatory local-part@domain. Not just @. edit: https://datatracker.ietf.org/doc/html/rfc2822#section-3.4.1 https://datatracker.ietf.org/doc/html/rfc2822#section-3.4.1
  - krageon 5y ago
    The RFC has a specified regex for validating email that absolutely nobody uses. Given that, matching on an @ (if you do it at all - even that is a bad idea. Just send an email) is the easiest approach that you're least likely to mess up.
    - hu3 5y ago
      Totally agree with just sending an email and see if it sticks. Perhaps tech giants could sponsor a new RFC for email addresses.
      - krageon 5y ago
        A tech giant sponsored RFC would just be what (US) tech giants feel is convenient and nothing else. I am vehemently opposed to them getting an even bigger say in how the internet works, it's going downhill fast enough as it is.
deleted 5y ago
[deleted]
kwhitefoot 5y ago
> the rules keep changing. No they don't. But they are sufficiently complicated that effectively the only way to validate an email address is to try sending a message to it and wait for a reply. Falsehoods that programmers believe: https://github.com/kdeldycke/awesome-falsehood#emails https://github.com/kdeldycke/awesome-falsehood#emails

mro_name 5y ago

> because the rules keep changing the single relevant rule is the same ever since: do an opt in. The spelling of the email address is of zero relevance and must not be nosed at.

Semaphor 5y ago
You forget one thing: People make typos. Assuming Sony blocks all LEN(TLD)>3, do you think they’ll lose more customers than they’ll lose from them not getting their validation mail because they typoed their mail? I’d bet that the first group is far smaller than the 2nd. Now instead of blocking it would be nice if they just asked if you are really sure that’s your email, but it’s probably still better than just accepting everything and trying to send a confirmation, both from a business and from a customer-friendliness perspective.
- mro_name 5y ago
  haven't forgotton. Typos happen not only in the tld. So check and warn if you will. But indeed let it happen and don't gag people. You need the opt-in and have people confirm or repeat in case.
amerkhalid 5y ago
Ran into this issue at one of the job. Developers wanted to implement validation based on RFCs. But QA and business refused to believe RFC rules and instead wanted their own rules. Guess what we end up implementing.
- Freak_NL 5y ago
  What did you end up doing? Anything more than checking if the string contains @ is wrong, and you should be able to find dozens of articles supporting this argument by now.
  - amerkhalid 5y ago
    We checked for dot in addition to @. There was probably also check for special chars which makes sense, iirc. One win we had was, we didn't hardcode any tlds.
- mro_name 5y ago
  even a RFC doesn't guarantee the existence of a mailbox - if the address e.g. has a typo in the name. And own rules very much sounds like management ever wanted to invent something.

viraptor 5y ago

> The assumption that .pizza is not a valid TLD was valid before, Why do you think so? We didn't have a strict set of TLDs before as far as I know. We just stagnated for a long time. But the list of TLDs has changed many times with countries being created/renamed. If you saw .pizza or .zx, systems should've behaved the same - tried resolving.

chockchocschoir 5y ago
> Why do you think so? Because .pizza wasn't a valid TLD before 2014 when it got delegated? Don't know what to tell you, .pizza haven't always existed, it's a relatively new gTLD that got introduced by the new gTLD program by ICANN. But I agree with you in general, trying to validate emails with regex will never work in practice, and has lots of issues. Only real way to validate a address is to actually send email, ask user to input data from said email.
- roel_v 5y ago
  "trying to validate emails with regex will never work in practice" Of course it does. It just depends on how you define 'validate emails'. Your statement is ambiguous. The Friedl has an entire appendix on a single regex that validates an email address to the format described in RFC 5322, although of course maybe a proper parser would be better, from a technical perspective. However, an email address being in the right format does not ensure that the user that email address is 'associated' with (for whatever definition of 'associated' that is appropriate to the application we're talking about) can/will actually receive emails send to that address. You should do both. Do a first 'pass' validation to check the user didn't make a type. This is more for user convenience, as this can give immediate feedback. And then send an actual email to see if email is deliverable there. The problem here is not that they checked the format. The problem is that they check the wrong way. Although it's not entirely unthinkable that it's sometimes appropriate to add additional constraints to which addresses you want to accept, e.g. banning certain tld's, or even only accepting whitelisted ones. If your fraud stats say that doing that reduces loss, it's not unreasonable to add such a third filter layer.
- viraptor 5y ago

jeroenhd 5y ago

The rules never changed, the TLD list did. The problem is that people like to put restrictions on which emails are "normal" and which are not. I wouldn't be surprised if websites like these only accept Gmail and Outlook email addresses with their nonsensical restrictions. Blocking an entire TLD because of fraud is insanity. There are plenty of realistic looking domains available for cheap in the dot com TLD, no need to go for dot pizza or dot dev to bypass spam filters.

dhosek 5y ago
>Blocking an entire TLD because of fraud is insanity. Wikipedia does this with .xyz links. When I asked about this (I wanted to put a link to finl.xyz on my user page), I was told that this was because of link spamming.
- Avamander 5y ago
  .xyz gets your spam score increased as well by a lot of mail filters. Price and stricter TLD rules do work as a deterrence to abuse.
  - Brian_K_White 5y ago
    Meanwhile xyz is a perfectly legit tld for sites about 3d printing, cnc machining, 3d modelling, etc. But get stigmatized and prejudged through no fault of their own. It's like name racism.

Aissen 5y ago

The rules never changed. One should only match if the email contains "@" (or better, use browser email inputs), and do the verification by sending an email to it (as you said). Validating TLDs was never a good idea, and still isn't. Yes, you can put UX helpers for spelling mistakes on most common domains (gmail…) (helps the majority), but never make it a hard error. It's that simple.

hu3 5y ago
IIRC the RFCs for email address specify mandatory local-part@domain. Not just @. edit: https://datatracker.ietf.org/doc/html/rfc2822#section-3.4.1 https://datatracker.ietf.org/doc/html/rfc2822#section-3.4.1
- krageon 5y ago
  The RFC has a specified regex for validating email that absolutely nobody uses. Given that, matching on an @ (if you do it at all - even that is a bad idea. Just send an email) is the easiest approach that you're least likely to mess up.
  - hu3 5y ago
    Totally agree with just sending an email and see if it sticks. Perhaps tech giants could sponsor a new RFC for email addresses.
    - krageon 5y ago
      A tech giant sponsored RFC would just be what (US) tech giants feel is convenient and nothing else. I am vehemently opposed to them getting an even bigger say in how the internet works, it's going downhill fast enough as it is.

deleted 5y ago

[deleted]

kwhitefoot 5y ago

> the rules keep changing. No they don't. But they are sufficiently complicated that effectively the only way to validate an email address is to try sending a message to it and wait for a reply. Falsehoods that programmers believe: https://github.com/kdeldycke/awesome-falsehood#emails https://github.com/kdeldycke/awesome-falsehood#emails