8 ms·

RFCs for email addresses are cool, but on the web we have our own standards! https://html.spec.whatwg.org/multipage/input.html#valid-e-mail-address https://htm

by dchest 4y ago

RFCs for email addresses are cool, but on the web we have our own standards! https://html.spec.whatwg.org/multipage/input.html#valid-e-mail-address https://html.spec.whatwg.org/multipage/input.html#valid-e-ma... "This requirement is a willful violation of RFC 5322, which defines a syntax for email addresses that is simultaneously too strict (before the "@" character), too vague (after the "@" character), and too lax (allowing comments, whitespace characters, and quoted strings in manners unfamiliar to most users) to be of practical use here." The regex is: /^[a-zA-Z0-9.!#$%&'*+\/=?^_`{|}~-]+@[a-zA-Z0-9](?:[a-zA-Z0-9-]{0,61}[a-zA-Z0-9])?(?:\.[a-zA-Z0-9](?:[a-zA-Z0-9-]{0,61}[a-zA-Z0-9])?)*$/ Every browser implements this regex for <input type="email">. Chromium: https://source.chromium.org/chromium/chromium/src/+/main:third_party/blink/renderer/core/html/forms/email_input_type.cc;l=46-53;bpv=0;bpt=1 https://source.chromium.org/chromium/chromium/src/+/main:thi... WebKit: https://github.com/WebKit/WebKit/blob/0d7afc5a45c140c44497a81e92416f01306be877/Source/WebCore/html/EmailInputType.cpp#L38 https://github.com/WebKit/WebKit/blob/0d7afc5a45c140c44497a8... Yes, do verify email addresses by sending a confirmation link if you bind users to their email addresses, though. Don't confuse validation with verification.

seanw444 4y ago
I like your final sentence. Very to-the-point.
hathawsh 4y ago
I tried using that expression for a while, but then a user with a valid email address containing upper unicode characters showed up. I switched to a simpler expression: ^[^@\s\x00-\x1f]+@[^@\s\x00-\x1f.]+(:?\.[^@\s\x00-\x1f.]+)*$ It requires exactly one "@", disallows whitespace and control characters, prevents repeated dots in the domain name, and ensures the domain doesn't end with a dot. It catches a few typos and I think it allows every real email address I've heard of.
- easrng 4y ago
  Domains ending with a dot are valid though, and it's needed sometimes. For example, someone@ai. (ai. is a TLD) is a different email than someone@ai (ai is a local hostname)
  - hathawsh 4y ago
    Good to know. I guess I'll simplify again. Besides, I wonder whether the expression is vulnerable to catastrophic backtracking.
  - Beltalowda 4y ago
    > it's needed sometimes When is this needed in the context of a Rails app? You never want to send emails to local hosts, and emails directly on the TLD is possible, but very little will work with them in practice.
    - turminal 4y ago
      > You never want to send emails to local hosts That's not entirely true, it depends on how your mailing infrastructure is set up.
      - Beltalowda 4y ago
        But in the context of a web app though? That's the key part: you're taking in email addresses from myapp.example.com. For other cases: sure, local email addresses can be useful (not often these days, but sometimes still are). But in this context I'm not really seeing any use case.
        bbatha 4y ago
        At previous $work, I wrote a rails app that managed releases. On the app you could sign up for email notifications when your change was released. These went through the unix local email system. So yes there is very much a use case
        lolinder 4y ago
        It's still a bit silly to tell every SaaS developer that they shouldn't validate that they have a normal-looking domain because some people's use cases call for sending emails to local addresses or to TLDs directly. If a developer needs to handle those cases, they know already, and they're in a vanishingly small minority.
- daurnimator 4y ago
  I invite you to try my test suite at https://github.com/daurnimator/lpeg_patterns/blob/master/spec/email_spec.lua https://github.com/daurnimator/lpeg_patterns/blob/master/spe...
- qwerty456127 4y ago
  > containing upper unicode characters Are e-mail addresses case-sensitive? I would always lowercase&trim a string meant to represent an e-mail address (or a domain name) before doing anything else with it.
  - electroly 4y ago
    The local-part before the @ can be case-sensitive, yes. I've never heard of a mail server that treats it as such, but the RFC says it's case sensitive.
    - deleted 4y ago
      [deleted]
  - jeroenhd 4y ago
    Not all uppercase characters have just one lowercase representation, and not all lowercase characters have an uppercase character. If you try to make the addresses case insensitive, you're bound to run into unnecessary complexities at some point. Email addresses aren't case insensitive in nature, but many commercial mail services (and misguided developers) assume they are.
  - hathawsh 4y ago
    To be clear, by "upper" unicode characters I mean non-ASCII international characters, not uppercase characters. For example, 我買@屋企.香港 is now a valid email address. [1] 1. https://en.wikipedia.org/wiki/Email_address#Internationalization_examples https://en.wikipedia.org/wiki/Email_address#Internationaliza...
- bobbylarrybobby 4y ago
  I'm pretty sure email addresses are allowed to have multiple @'s. I believe everything after the first is (supposed to be) considered part of the domain.
  - chrismorgan 4y ago
    Domain labels cannot contain @. If you support comments or quoted strings (e.g. (@)."@"@example) then you can get multiple at signs in an address, but in practice their only use these days is attacks where you confuse a badly-written server, which is part of why the web actively decided to disallow them.
    - eadmund 4y ago
      > in practice their only use these days is attacks where you confuse a badly-written server Or, you know, if you actually want to use an at sign in the manner allowed by the RFCs. That's the point of standards: if something is allowed, then … it is allowed. Refusing to accept behaviour permitted by the standard is just broken.
      - chrismorgan 4y ago
        Usage is what matters. Standards documents only have value as long as people follow them; once enough people deviate, they cease to have meaning. This is thoroughly the case with a lot of stuff pertaining to email: if you want to parse arbitrary emails, for example, the relevant MIME RFCs are altogether insufficient, because too many people have done it incorrectly de jure, often to the point where it’s no longer incorrect de facto because everyone supports the deviation. (This is a deviation from spec in the direction of permission.) By hearsay alone, I say: comments and quoted strings in email addresses are, de facto, more or less dead. Comments especially, which were intended just as a compatibility measure for stuff from over forty years ago (when this was being developed organically, before there was any real spec). Most software whose direct business is email (that is, MUAs and MTAs) will more or less support them for historical reasons, but newer MUAs are likely not to support them in full (again especially comments), and I suspect the considerable majority of other software that deals in email addresses won’t support one or both of them. (This is a deviation from spec in the direction of restriction.)
omegalulw 4y ago
What is the significance of [a-zA-Z0-9-]{0,61}? Some special limit on 61 characters?
- Beltalowda 4y ago
  Domain labels are limited to 63 bytes, with the [a-z..] before and after that makes 61 + 1 + 1 = 63.
  - deleted 4y ago
    [deleted]
gpvos 4y ago
Oh wow, that is awful.
boplicity 4y ago
If you really want to take validation seriously, why not add a human element to it, and not design one line of code to try to fix everything? One of the biggest problems with email input is typos -- and there are some very common typos that could easily be accounted for with code. For example foo@gmail.co, foo@gmial.com, foo@comcast, etc. It should be common, when these types of typos occur, to prompt the user to fix them. Unfortunately, this is quite rare.
- tough 4y ago
  Everytime I input my .me domain in one of those well intended websites I get nagged about maybe being wrong and wantint to use .de or whatever. No, I know my email, thx, it's bein prefilled from auto-complete. Don't fucking tell my Im typing it wrong when 1. Its my email, and I'm not even typing
  - grimjack00 4y ago
    I have the same issues with my .name domain. Most of the reason I have a gmail address as well is for the sites that don't think "first@last.name" is a real address.
  - alasdair_ 4y ago
    Related: I used to use me@<myname>.com but gave up because Gmail uses "Me" as the name when displaying a user's own email address and it was confusing people enormously because a thread would look like: >>>Me wrote: >>Me wrote: >Me wrote: Me wrote:
- throwaway787544 4y ago
  That would imply that programmers care about users. I've never met a single programmer who cared what the user experience was like. Today they even brag about this, saying "I only want to care about my code!" I see it here on HN all the time, and I hear it in companies.
  - jve 4y ago
    The other thing is overengineering. Spending a week to implement email validation? Maybe, if it is REALLY important and results in lost revenue. Otherwise I don't know what common typos are. I would have to do research on that. Then I have popular local email providers where I may come up with typos. but then again, it is only in my region - I would have to get some list of popular email providers in every country. Come up with a way to update that list after few years when new email providers come out. Or maybe develop a solution that matches "close enough" entries in my pre-defined list. UX effort could be spent in an area which is used more often than only once. If there is a library for that, yeah, then slap it in, UX has been upgraded in no time. And again, only if revenue is on the other side of consideration, the development effort pays off.
thrwyoilarticle 4y ago
In fact, doesn't 'RFC' imply that it isn't a standard? A pragmatic, domain-specific spec beats that. edit: there's more nuance than that. Some RFCs are standards. 2822 is 'Category: Standards Track', so is a standard if referred to by its STD number? I can't find what its STD number is.
goto11 4y ago
This is the sane approach. Nobody uses the bizarre address formats with quoted strings and embedded comments and whatnot. There are included in the RFC because of backwards compatibility with legacy email systems from before SMTP.
- jeroenhd 4y ago
  Quoted strings are actually used in the real world, though they're certainly not common. Shitty front ends are definitely putting an end to that, though.
  - xeromal 4y ago
    I'd say that's for the better.
chrismorgan 4y ago
One important aspect of this that I often see people forgetting is that this check is designed for working with ASCII, but the domain name at least can be non-ASCII. User interfaces should remember to support IDN in domain labels and convert it to punycode before validating, and if you store A-labels (which you probably do) then convert it back to IDN form when presenting it to users. (Alas, <input type=email> still doesn’t support non-ASCII in the local part, which isn’t supported everywhere but is, I believe, fairly widely supported now. See https://github.com/whatwg/html/issues/4562 https://github.com/whatwg/html/issues/4562 plus https://en.wikipedia.org/wiki/Email_address_internationalization https://en.wikipedia.org/wiki/Email_address_internationaliza... for a little more background on what it is.)
- 411111111111111 4y ago
  I thought they've rolled back most of this because it made phishing domains indistinguishable from the real ones? I used an-emoji.my.domain for a while until chrome changed it back to punycode
  - chrismorgan 4y ago
    I erred slightly in what I wrote there; I said IDN form for user display, but should have said IDNA form, which is the set of restrictions to mitigate those hazards. That will allow things like उदाहरण.example, while leaving emoji as xn--n3h.example since they’re not valid in IDNA. See also https://en.wikipedia.org/wiki/Internationalized_domain_name https://en.wikipedia.org/wiki/Internationalized_domain_name and https://en.wikipedia.org/wiki/Emoji_domain https://en.wikipedia.org/wiki/Emoji_domain.
    - chrismorgan 4y ago
      Actually, I’ve got to add another note because my memory and experience was insufficient here: see also https://en.wikipedia.org/wiki/IDN_homograph_attack#Client-side_mitigation https://en.wikipedia.org/wiki/IDN_homograph_attack#Client-si... for descriptions of what further restrictions browsers do, with the most notable additional filter being disallowing mixing scripts. This is something that would be nice to formalise in some way, though I don’t know of any venue suitable for the task.

by dchest 4y ago

seanw444 4y ago

I like your final sentence. Very to-the-point.

hathawsh 4y ago

I tried using that expression for a while, but then a user with a valid email address containing upper unicode characters showed up. I switched to a simpler expression: ^[^@\s\x00-\x1f]+@[^@\s\x00-\x1f.]+(:?\.[^@\s\x00-\x1f.]+)*$ It requires exactly one "@", disallows whitespace and control characters, prevents repeated dots in the domain name, and ensures the domain doesn't end with a dot. It catches a few typos and I think it allows every real email address I've heard of.

easrng 4y ago
Domains ending with a dot are valid though, and it's needed sometimes. For example, someone@ai. (ai. is a TLD) is a different email than someone@ai (ai is a local hostname)
- hathawsh 4y ago
  Good to know. I guess I'll simplify again. Besides, I wonder whether the expression is vulnerable to catastrophic backtracking.
- Beltalowda 4y ago
  > it's needed sometimes When is this needed in the context of a Rails app? You never want to send emails to local hosts, and emails directly on the TLD is possible, but very little will work with them in practice.
  - turminal 4y ago
    > You never want to send emails to local hosts That's not entirely true, it depends on how your mailing infrastructure is set up.
    - Beltalowda 4y ago
      But in the context of a web app though? That's the key part: you're taking in email addresses from myapp.example.com. For other cases: sure, local email addresses can be useful (not often these days, but sometimes still are). But in this context I'm not really seeing any use case.
      - bbatha 4y ago
        At previous $work, I wrote a rails app that managed releases. On the app you could sign up for email notifications when your change was released. These went through the unix local email system. So yes there is very much a use case
        lolinder 4y ago
        It's still a bit silly to tell every SaaS developer that they shouldn't validate that they have a normal-looking domain because some people's use cases call for sending emails to local addresses or to TLDs directly. If a developer needs to handle those cases, they know already, and they're in a vanishingly small minority.
daurnimator 4y ago
I invite you to try my test suite at https://github.com/daurnimator/lpeg_patterns/blob/master/spec/email_spec.lua https://github.com/daurnimator/lpeg_patterns/blob/master/spe...
qwerty456127 4y ago
> containing upper unicode characters Are e-mail addresses case-sensitive? I would always lowercase&trim a string meant to represent an e-mail address (or a domain name) before doing anything else with it.
- electroly 4y ago
  The local-part before the @ can be case-sensitive, yes. I've never heard of a mail server that treats it as such, but the RFC says it's case sensitive.
  - deleted 4y ago
    [deleted]
- jeroenhd 4y ago
  Not all uppercase characters have just one lowercase representation, and not all lowercase characters have an uppercase character. If you try to make the addresses case insensitive, you're bound to run into unnecessary complexities at some point. Email addresses aren't case insensitive in nature, but many commercial mail services (and misguided developers) assume they are.
- hathawsh 4y ago
  To be clear, by "upper" unicode characters I mean non-ASCII international characters, not uppercase characters. For example, 我買@屋企.香港 is now a valid email address. [1] 1. https://en.wikipedia.org/wiki/Email_address#Internationalization_examples https://en.wikipedia.org/wiki/Email_address#Internationaliza...
bobbylarrybobby 4y ago
I'm pretty sure email addresses are allowed to have multiple @'s. I believe everything after the first is (supposed to be) considered part of the domain.
- chrismorgan 4y ago
  Domain labels cannot contain @. If you support comments or quoted strings (e.g. (@)."@"@example) then you can get multiple at signs in an address, but in practice their only use these days is attacks where you confuse a badly-written server, which is part of why the web actively decided to disallow them.
  - eadmund 4y ago
    > in practice their only use these days is attacks where you confuse a badly-written server Or, you know, if you actually want to use an at sign in the manner allowed by the RFCs. That's the point of standards: if something is allowed, then … it is allowed. Refusing to accept behaviour permitted by the standard is just broken.
    - chrismorgan 4y ago
      Usage is what matters. Standards documents only have value as long as people follow them; once enough people deviate, they cease to have meaning. This is thoroughly the case with a lot of stuff pertaining to email: if you want to parse arbitrary emails, for example, the relevant MIME RFCs are altogether insufficient, because too many people have done it incorrectly de jure, often to the point where it’s no longer incorrect de facto because everyone supports the deviation. (This is a deviation from spec in the direction of permission.) By hearsay alone, I say: comments and quoted strings in email addresses are, de facto, more or less dead. Comments especially, which were intended just as a compatibility measure for stuff from over forty years ago (when this was being developed organically, before there was any real spec). Most software whose direct business is email (that is, MUAs and MTAs) will more or less support them for historical reasons, but newer MUAs are likely not to support them in full (again especially comments), and I suspect the considerable majority of other software that deals in email addresses won’t support one or both of them. (This is a deviation from spec in the direction of restriction.)

omegalulw 4y ago

What is the significance of [a-zA-Z0-9-]{0,61}? Some special limit on 61 characters?

Beltalowda 4y ago
Domain labels are limited to 63 bytes, with the [a-z..] before and after that makes 61 + 1 + 1 = 63.
- deleted 4y ago
  [deleted]

gpvos 4y ago

Oh wow, that is awful.

boplicity 4y ago

If you really want to take validation seriously, why not add a human element to it, and not design one line of code to try to fix everything? One of the biggest problems with email input is typos -- and there are some very common typos that could easily be accounted for with code. For example foo@gmail.co, foo@gmial.com, foo@comcast, etc. It should be common, when these types of typos occur, to prompt the user to fix them. Unfortunately, this is quite rare.

tough 4y ago
Everytime I input my .me domain in one of those well intended websites I get nagged about maybe being wrong and wantint to use .de or whatever. No, I know my email, thx, it's bein prefilled from auto-complete. Don't fucking tell my Im typing it wrong when 1. Its my email, and I'm not even typing
- grimjack00 4y ago
  I have the same issues with my .name domain. Most of the reason I have a gmail address as well is for the sites that don't think "first@last.name" is a real address.
- alasdair_ 4y ago
  Related: I used to use me@<myname>.com but gave up because Gmail uses "Me" as the name when displaying a user's own email address and it was confusing people enormously because a thread would look like: >>>Me wrote: >>Me wrote: >Me wrote: Me wrote:
throwaway787544 4y ago
That would imply that programmers care about users. I've never met a single programmer who cared what the user experience was like. Today they even brag about this, saying "I only want to care about my code!" I see it here on HN all the time, and I hear it in companies.
- jve 4y ago
  The other thing is overengineering. Spending a week to implement email validation? Maybe, if it is REALLY important and results in lost revenue. Otherwise I don't know what common typos are. I would have to do research on that. Then I have popular local email providers where I may come up with typos. but then again, it is only in my region - I would have to get some list of popular email providers in every country. Come up with a way to update that list after few years when new email providers come out. Or maybe develop a solution that matches "close enough" entries in my pre-defined list. UX effort could be spent in an area which is used more often than only once. If there is a library for that, yeah, then slap it in, UX has been upgraded in no time. And again, only if revenue is on the other side of consideration, the development effort pays off.

thrwyoilarticle 4y ago

In fact, doesn't 'RFC' imply that it isn't a standard? A pragmatic, domain-specific spec beats that. edit: there's more nuance than that. Some RFCs are standards. 2822 is 'Category: Standards Track', so is a standard if referred to by its STD number? I can't find what its STD number is.

goto11 4y ago

This is the sane approach. Nobody uses the bizarre address formats with quoted strings and embedded comments and whatnot. There are included in the RFC because of backwards compatibility with legacy email systems from before SMTP.

jeroenhd 4y ago
Quoted strings are actually used in the real world, though they're certainly not common. Shitty front ends are definitely putting an end to that, though.
- xeromal 4y ago
  I'd say that's for the better.

chrismorgan 4y ago

One important aspect of this that I often see people forgetting is that this check is designed for working with ASCII, but the domain name at least can be non-ASCII. User interfaces should remember to support IDN in domain labels and convert it to punycode before validating, and if you store A-labels (which you probably do) then convert it back to IDN form when presenting it to users. (Alas, <input type=email> still doesn’t support non-ASCII in the local part, which isn’t supported everywhere but is, I believe, fairly widely supported now. See https://github.com/whatwg/html/issues/4562 https://github.com/whatwg/html/issues/4562 plus https://en.wikipedia.org/wiki/Email_address_internationalization https://en.wikipedia.org/wiki/Email_address_internationaliza... for a little more background on what it is.)

411111111111111 4y ago
I thought they've rolled back most of this because it made phishing domains indistinguishable from the real ones? I used an-emoji.my.domain for a while until chrome changed it back to punycode
- chrismorgan 4y ago
  I erred slightly in what I wrote there; I said IDN form for user display, but should have said IDNA form, which is the set of restrictions to mitigate those hazards. That will allow things like उदाहरण.example, while leaving emoji as xn--n3h.example since they’re not valid in IDNA. See also https://en.wikipedia.org/wiki/Internationalized_domain_name https://en.wikipedia.org/wiki/Internationalized_domain_name and https://en.wikipedia.org/wiki/Emoji_domain https://en.wikipedia.org/wiki/Emoji_domain.
  - chrismorgan 4y ago
    Actually, I’ve got to add another note because my memory and experience was insufficient here: see also https://en.wikipedia.org/wiki/IDN_homograph_attack#Client-side_mitigation https://en.wikipedia.org/wiki/IDN_homograph_attack#Client-si... for descriptions of what further restrictions browsers do, with the most notable additional filter being disallowing mixing scripts. This is something that would be nice to formalise in some way, though I don’t know of any venue suitable for the task.