14 ms·
Perfect email regex finally found
- andrewvc 16y agoHow ugly do non-regex based email validation functions look? I've never seen one, but I've always wondered if that was a more elegant solution.
- alnayyir 16y agoRegex is the only real sensible way to validate strings until something better is found. Even if you just wrote code to do it manually, you'd really just be writing a verbose and poorly implemented finite state machine that globbed symbols together, which in the end, would just be inferior to writing a well tested Regex string. Regex can be easier to read if you have something do a graphical expansion for you. Otherwise, it's write-once, read-never.
- adamc 16y agoThat's just silly. Depending on what's in the string, writing a parser might be much better than a regex. Lots of parser libraries already out there, too.
- Ixiaus 16y agoI can agree with this sentiment, there's also nothing wrong with writing a parser that breaks the input up into pieces and validates those individual parts using specialized regular expressions.
- aphyr 16y agoEspecially when the spec itself is in EBNF.
- _delirium 16y agoTrue, though it's EBNF with a bunch of explanatory text and annotations. What would be interesting, but I can't find with some googling: Has someone implemented a parser-generator based on the spec? The ideal would be that the parser specification looks a lot like the RFC, since then you'd have more confidence it was actually correct (and it'd be easier to maintain for future changes).
- aphyr 16y agoMy comment above is a regex derived straightforwardly from the spec. Not a full parser, though.
- twopoint718 16y agoThere was a long post on the topic of 'regex-vs-parser' for email on reddit a while back. I hope I'm pointing to the correct person, but he wrote a parser in Haskell that validates against RFC5322: http://hackage.haskell.org/package/email-validate http://hackage.haskell.org/package/email-validate
- alnayyir 16y agoBy all means, use a CFG or a PEG for parsing emails. While you do that, I'll do something that doesn't involve tweezers and code.
- deleted 16y ago[deleted]
- JadeNB 16y ago> Regex is the only real sensible way to validate strings until something better is found. There are plenty of ‘better’ (in the sense of ‘more powerful’) string-validation techniques. For example, lots of grammars are expressed in BNF; the languages that can be so expressed are (if I remember my Chomsky hierarchy correctly) the context-free grammars, a strictly larger class than the regular languages. The extra power comes from the fact that they have the expressive power of a finite-state automaton augmented by an (infinite) stack. (It's fair to argue that it's not ‘really’ infinite, since a computer's memory is finite; but, in that sense, real-life computers will never be Turing complete.) (Of course, common ‘regular-expression’ libraries aren't actually regular any more, because of added features like capture groups. I don't know if they recognise all CFG's, though; I suspect not.) Given this, why would we use regular expressions? Well, by intentionally sacrificing power, we can achieve faster matching (http://swtch.com/~rsc/regexp/regexp1.html http://swtch.com/~rsc/regexp/regexp1.html) and, probably, lower memory usage. Sometimes this trade-off is worth it, even if it means that the match must be somewhat fuzzy; but sometimes one needs a precise match, and regexes just aren't up to the job.
- huherto 16y ago> Given this, why would we use regular expressions? If I can get a way with a simple regexp I will use it. If it starts getting complicated I would use something else.(e.g. top down parser or a combination)
- JadeNB 16y ago> If I can get a way with a simple regexp I will use it. If it starts getting complicated I would use something else.(e.g. top down parser or a combination) Isn't that what I said?
- huherto 16y ago(If understood correctly) You based it in speed and memory efficiency. I rather base it on having something simple and readable.
- huherto 16y agoIf I remember correctly my theory of languages course a parser (implemented in a turing machine) is more powerful than a reggex (which is a finite state machine).
- compay 16y agoRoberto Ierusalimschy of Lua fame often uses this example of how PEGs (parsing expression grammars) can be much easier than regexps for some tasks. http://lua-users.org/lists/lua-l/2009-10/msg00817.html http://lua-users.org/lists/lua-l/2009-10/msg00817.html
- FlorinAndrei 16y agoCompared to regex, by definition, nothing can be uglier.
- deutronium 16y agoOne nice looking trick I've seen is checking the domain of the email address for an MX record. (e.g. http://php.dzone.com/news/php-email-validator-email-mx-d http://php.dzone.com/news/php-email-validator-email-mx-d) Possibly this approach is too slow to use by itself without a regex. Also maybe there are other problems with this method I'm not aware of?
- drivebyacct 16y agoI suggested that a while back but was told that due to spam, you can get lots of false negatives from this technique and others. The best way to validate an email remains to be a test email apparently.
- pkulak 16y agoYeah, that's a really quick way to get IP banned from most email hosts.
- spicyj 16y agoYou must be thinking of something like this: http://www.webdigi.co.uk/blog/2009/how-to-check-if-an-email-address-exists-without-sending-an-email/ http://www.webdigi.co.uk/blog/2009/how-to-check-if-an-email-... Merely looking up MX records is something you do for a _domain_ and it's no different than the DNS requests your computer makes when you go to a website.
- pornel 16y agoMX record is not required. You should allow hosts with only A records too. http://tools.ietf.org/html/rfc5321#section-5.1 http://tools.ietf.org/html/rfc5321#section-5.1
- curio 16y agopractically speaking, most respectable email services will have an MX record. i tested it on about 30,000 emails & a simple MX lookup caught about 40 gmial's and only 1 false positive.
- deleted 16y ago
- DanBlake 16y ago&*=?^+{}'~@12.34.56.78:2000 really looks strange, even though its a valid email. Good luck trying to register on any site with it though :)
- petercooper 16y agoWhat's the deal with using a port number in an e-mail address? I can't imagine many systems supporting that. Anyone got any more info on it that doesn't involve me digging through 101 pages of RFCs? :-)
- tedunangst 16y agoI don't think that's valid. I think they misread the RFC. First of all, an IP address (address-literal) is supposed to be enclosed in [square brackets]. The colon is only allowed to designate address family, as in "foo@[IPv6:XXXXXX]".
- pornel 16y agoNot perfect: doesn't support IDN without punycode. Doesn't support IDN TLDs at all. Users of http://موقع.وزارة-الاتصالات.مصر http://موقع.وزارة-الاتصالات.مصر won't be pleased :)
- maw 16y agoAt the risk of sounding flip, not supporting punycode sounds like a feature. Getting internationalization right is clearly important, but punycode as a means of doing so? It's one of the many things that make me weep for my industry. In fact, I often wonder if punycode is a prank that got out of hand. UTF-8, on the other hand, would have been excellent for this purpose.
- pornel 16y agoDNS doesn't allow 8-bit characters, so UTF-8 is not an option. I think punycode is more efficient than UTF-5 would be.
- maw 16y agoTrue. It doesn't. But as far as I know, there is no deep technical reason why it can't. The dots in domain names are '\0', and a few characters ('@', in particular) need to remain reserved, but other than that, what stops UTF-8 from being used?
- gojomo 16y agoNothing is finished, nothing is permanent, and nothing is perfect. In particular, one of the evaluation tests used here is wrong: it requires failure-to-match on a TLD with a digit in it: numbersInTLD@domain.c0m In fact, IDN TLDs will have digits in them. An internet-draft is in the works to replace RFC1123's IDN-unfriendly implication that digits in TLDs are illegal: http://tools.ietf.org/html/draft-liman-tld-names-02 http://tools.ietf.org/html/draft-liman-tld-names-02
- sigzero 16y agoIf you say "will have" and "in the works" means it isn't the standard and the current test is valid.
- gojomo 16y agoA nice try at pedantry, which I would usually respect, but in the domain of internet standards with which I familiar, you are wrong. The existing specs are in conflict, with the more recent ones (such as IDN) allowing digits. Internet authorities, including ICANN, have enabled domains with digits in TLDs; software which is far more foundational than any web-app's email validation regex has been updated. Registration of names in some of these digited-TLDs has begun; you can visit these TLDs with your browser; your users can have functioning email addresses on these TLDs. If your app rejects such email addresses because of slavish compliance with imprecise language in a 31-year-old RFC, you'd be the one violating prevailing standards, which are a function of more than just formal IETF RFCs. That the Internet-Draft I referenced may soon become an RFC is just cleaning up loose ends on a change that's already happened. This final step isn't even strictly necessary for the de facto standard to have changed by consensus among practitioners. Plenty of vibrant well-understood standards never reach RFC status, nor pass through any formal standards body. The standard is ultimately what people do, not what someone once-upon-a-time decreed.
- th 16y agoThose hard-coded top-level domains in the "improved" regex may also be also an issue: http://en.wikipedia.org/wiki/Top-level_domain#Proposed_domains http://en.wikipedia.org/wiki/Top-level_domain#Proposed_domai...
- percept 16y agoI've accepted that it's best to treat people like grown-ups and if there's '@' and '.' and it's retyped then it passes. Someone can easily submit a fake name or phone number or street address, and e-mail's no different. If they get it wrong, intentionally or not, then they don't get their receipt, confirmation, validation link, etc. and I believe in most cases the incentive is there for them to get it right. In the rare case where there's some incentive to circumvent the system and this has some measurable impact on a site, then more validation may be warranted. Otherwise, why worry about it? Also: HTML5. ;)
- jbr 16y agoAlong those lines, I've settled on the following overly permissive regex: /^[^\s@]+@[^\s@]+\.[^\s@]{2,}$/ -- it makes sure it looks something like an email address (a@b.cd)
- jrockway 16y agoWhat if I want to receive mail directly at my TLD?
- DougBTX 16y agoInterestingly, there is a test for that in the article - under the "These should be invalid" section.
- _fool 16y agointerestingly i count 19 TLD's with mx records. but judging from your earlier post you know as well as i do (i use <single character>@w.tf for many purposes) that there are pretty much no web forms on the planet that would take such an address. tis a shame, but such addresses while pretty nifty are also frequently unusable. not to mention trying to explain to someone that that is really your email address and if it doesn't work they should talk to their ISP/mail client author/etc...well, suffice it to say they won't be getting in touch with you that way anyhow. (be that a blessing or curse, it's just reality)
- davidu 16y ago
- Periodic 16y agoIt does not look like any of these can detect or were tested against quoted-local-part addresses. As I understand it, the local part can be quoted to allow illegal characters to be used, e.g. "John Doe"@example.com I fully understand that these are not in common use, but they are part of the RFC and may be in use somewhere.
- Terretta 16y agoHis test also considers the % sign invalid, yet I've both sent and received email that required the % for the mail to be routed/relayed correctly (think "in care of" or c/o). Granted, that was 17 years ago, but who's to say it's not in use somewhere?
- defdac 16y agoFascination. Then acid reflux.
- pavel_lishin 16y agoI believe that the image you're looking for to express your current emotional state is http://uploads.postfarm.net/public/postfarm/uploads-2.0/i/ifrewup.jpg http://uploads.postfarm.net/public/postfarm/uploads-2.0/i/if...
- fragmede 16y agoIt thinks baddomain@300.0.0.1 is valid, and thinks decimaldomain@2130706433 (localhost) is not valid. It also thought user@3ffe:1900:4545:3:200:f8ff:fe21:67cf as invalid.
- gry 16y agoI've accepted email validation is like security, better left to libraries by people smarter than I am...TMail with over 2000 test cases. But we know a valid email address when we see it, right? -- Edit: Nicely done, HTML5.
- generalk 16y agoI never understood the purposes of the "perfectly valid by the RFC" email regex. You may be able to with 100% accuracy say that something should be an email address, but you'll never be able to tell if it's a valid account on the server, or if the server even exists.
- edw519 16y agoAs a typical optimizer, I was always trying to reduce my source code a few more bytes or speed up my processes a few nanaseconds here and there. I was so proud of myself. Until I tried to maintain my own slickness. There's a fine line between clever and practical. Why do I have a gut feeling that this approach is way over that line?
- aphyr 16y agoIt's certainly more concise than my previous favorite, qtext = '[^\\x0d\\x22\\x5c\\x80-\\xff]' dtext = '[^\\x0d\\x5b-\\x5d\\x80-\\xff]' atom = '[^\\x00-\\x20\\x22\\x28\\x29\\x2c\\x2e\\x3a-' + '\\x3c\\x3e\\x40\\x5b-\\x5d\\x7f-\\xff]+' quoted_pair = '\\x5c[\\x00-\\x7f]' domain_literal = "\\x5b(?:#{dtext}|#{quoted_pair})*\\x5d" quoted_string = "\\x22(?:#{qtext}|#{quoted_pair})*\\x22" domain_ref = atom sub_domain = "(?:#{domain_ref}|#{domain_literal})" word = "(?:#{atom}|#{quoted_string})" domain = "#{sub_domain}(?:\\x2e#{sub_domain})*" local_part = "#{word}(?:\\x2e#{word})*" addr_spec = "#{local_part}\\x40#{domain}" pattern = Regexp.new "\\A#{addr_spec}\\z", nil, 'n'
- billturner 16y agoThis is the same one I've been using the last couple of years, and I wish I could remember where I first came across it.
- dkubb 16y agoFrom here maybe? http://tfletcher.com/lib/rfc822.rb http://tfletcher.com/lib/rfc822.rb
- eli 16y agoUh, what happens if a new TLD has more than 6 characters? I don't get this preoccupation with making sure addresses look valid. The ONLY way to validate an email address is to send it a message.
- dkoch 16y agoThis one in perl is fun as well: http://www.ex-parrot.com/pdw/Mail-RFC822-Address.html http://www.ex-parrot.com/pdw/Mail-RFC822-Address.html
- perplexes 16y agoUm, UTF-8? Unicode domains? Regexing email is a waste of time.
- angelbob 16y agoIt makes me happy that a contest of this kind is hosted at the domain "fighting for a lost cause" :-)
- snprbob86 16y agoSo many web services do this wrong, that it isn't even worth doing it right: no one is going to complain that your service doesn't accept their "wacky! quoted"@email.address Sometimes, you aren't validating a whole string, you are searching for email addresses in a sea of text, or an arbitrarily delimited, user-entered list of contacts. Support usernames with alphanumerics, dashes, underscores, periods, and plus signs; Require a single @; Support domains with alphanumerics, dashes, and at least one period. Screw anyone with something more complex than that. Done deal.
- huherto 16y agoI see your point. If somebody is using something weird, they probably have problems everywhere.
- mey 16y agoI assume this breaks with the new non-latin TLDs or IPV6
- kapranoff 16y agoThere's a classic email regex from Jeffry Friedl (author of "Mastering Regular Expressions") himself: http://www.diablotin.com/librairie/autres/mre/chBB.html http://www.diablotin.com/librairie/autres/mre/chBB.html It's 6,598 bytes long.
- avar 16y agoHere's one using modern Perl regex features. It's part of the perl regression tests: http://github.com/mirrors/perl/blob/blead/t/re/reg_email.t#L13 http://github.com/mirrors/perl/blob/blead/t/re/reg_email.t#L...
- andrewcooke 16y agoI have an implementation of RFC3696 http://www.faqs.org/rfcs/rfc3696.html http://www.faqs.org/rfcs/rfc3696.html (which is the spec for validating emails) in Python here - http://www.acooke.org/lepl/api/lepl.apps.rfc3696-module.html http://www.acooke.org/lepl/api/lepl.apps.rfc3696-module.html That is part of Lepl - http://www.acooke.org/lepl/ http://www.acooke.org/lepl/ - and although it's implemented in a recursive decent parser, much is compiled to regular expressions for efficiency. So you get the best of all worlds: regexp efficiency; parser accuracy; standards based. A blog post on the compilation to regexps is here - http://www.acooke.org/cute/LEPLOptimi0.html http://www.acooke.org/cute/LEPLOptimi0.html
- jimmyrcom 16y agoThen they DDoS the login form with a long password and email, and between the md5 and the retarded regex, your cpu is choking. You still need to limit action by ip and add some kind of a captcha system. People can just invite themselves to gmail a million times. Email verification is not so useful anymore.
- bengross 16y agoI wrote an article "Validating Email Address in Web Forms - The Hazards of Complexity" that discusses several comprehensive email validation libraries and common problems with complex methods of validation. http://www.messagingnews.com/onmessage/ben-gross/validating-email-address-web-forms-hazards-complexity http://www.messagingnews.com/onmessage/ben-gross/validating-...
- jimfl 16y agoGreat! Now get to work on the perfect HTML regex. (I know.)
- lazyant 16y agoAFAIK a naked IP address as domain like IPInsteadOfDomain@127.0.0.1 doesn't work (at least in some mail servers like Postfix they are bounced with a "bad address syntax error). The address has to be like IPInsteadOfDomain@[127.0.0.1] and I don't see this case covered.
- javathehut 16y agoI'm so totally gonna steal this!
- nfnaaron 16y ago"Perfect email regex finally found" Based on the HN title I thought it was going to be an article describing a post-it found on Fermat's dressing table mirror. Instead it's a list of mostly correct regexes. As Miracle Max might have observed, mostly correct is partly imperfect, and partly imperfect is Not Perfect.
- mrb 16y agoNOT perfect, FAILS to accept valid addresses Another know-it-all web developer guy who thinks he got his regex right... These are valid addresses that he rejects: user@ua (.ua = Ukraine) user@km (.km = Comoros) user@ne (.ne = Niger) Many ccTLDs have MX or A records pointing to real MTAs.
- jemfinch 16y agoHow in the world did this get 172 points? After this I almost stopped paying attention: "It's my philosophy that it's better to accept a few invalid addresses than reject any valid ones, so I'm shooting for 0 false-positives and as few false-negatives as possible." But then I looked at the regexps and they miss an absolutely trivial fact: valid email addresses can end in a dot. "jemfinch@supybot.com." is just as valid (more so, in fact) than "jemfinch@supybot.com".
- pbhjpbhj 16y ago>How in the world did this get 172 points? Things don't have to be well done, nor do you have to agree with them for them to be worth consideration/stimulating.
- jholloway 16y agoI've been surprised at the number of forms that tell me my email address, which has a single-letter local part, is invalid.
- skoob 16y ago"Now you have two problems." Seriously, I can't think of a single good reason why you would want to check whether an email address is "valid". What you should be concerned about is whether or not the address works (and usually, can/does the person who just signed up actually read and reply to email to that address). Hypothetically, if an invalid address works (due to bugs in mail systems) -- then it works, and the only problem with accepting such an address is that the bugs might get fixed. If an address is valid, there's no reason to assume that it will work or that it belongs to the person who signed up. It isn't even a good way of detecting typos; transpose two characters in an email address and it will most likely still pass your validation.
- mvalle 16y agoSeriously, I can't think of a single good reason why you would want to check whether an email address is "valid"! Because it's fun.
- dkubb 16y agoI've often wondered why we don't have any standardized test cases for email validation. A simple list of known good, and known bad addresses to use as test cases would really help. You could almost make a game out of finding valid addresses that are not matched, or invalid addresses that are false positives, or optimizing candidate regexps. The list could be continuously growing as new variations are discovered, and tested against previously submitted candidate regexps.
- palish 16y agoWow. /^[-a-z0-9~!$%^&*_=+}{\'?]+(\.[-a-z0-9~!$%^&*_=+}{\'? ]+)*@([a-z0-9_][-a-z0-9_]*(\.[-a-z0-9_]+)*\.(aero|arpa| biz|com|coop|edu|gov|info|int|mil|museum|name|net|org| pro|travel|mobi|[a-z][a-z])|([0-9]{1,3}\.[0-9]{1,3}\. [0-9]{1,3}\.[0-9]{1,3}))(:[0-9]{1,5})?$/i I mean... Why do this? Just, why? It's almost unreadable. Just write a short 20-line function which validates an email address. Use if statements. Write comments. Then verify that your algorithm does in fact handle all corner cases, just like the regexp does. (Your verifications will be in the form of short, simple unit test case functions.) To encourage regexp abuse like that is to encourage bad programming.
- Groxx 16y ago(late, but...) Frequently because Perl's regex library is insanely fast. Faster than if statements + smaller regex / roll-your-own. (as long as there's no look-ahead / look-behinds. It's still fast then, but custom functions can sometimes do better.)
- deleted 16y ago[deleted]
- ryan-allen 16y agoI think a basic token based parser might be more appropriate for thorough validation of email addresses. I see it similar to trying to match XML or HTML with regexps, very common but so very easy to break. You can't go to wrong parsing an address token by token, can you?
- ck2 16y agoWell that's one less thing. Now someone please make the perfect JSON regex decoder :-)
- augustl 16y agoEmail validation is user input validation; you are protecting the user from some cases of erronous input, and yourself from stuff that doesn't even look like an email. I find these large catch-all email regexps silly for two reasons: 1. They are hard to write, hard to understand, and hard to maintain. 2. Most importantly, they are difficult to understand for users. "You entered an invalid email". Now what? the user asks. This is why e-mail validation should be done in steps. Here is some Rails pseudocode: validates_format_of :email, :with => /@/, :message => "Needs to contain an @." validates_format_of :email, :with => /\.[^\.]+$/, :message => "Has to end with .com, .org, .net, etc." validates_format_of :email, :with => /^.+@/, :message => "Must have an address before the @" validates_format_of :email, :with => /^[^@]+@[^@]+$/, :message => "Must be of the format 'something@something.xxx'" Much easier to write, much easier to maintain, and much better error messages to the users.
- tmountain 16y agoSomething tells me this might not match admin@παράδειγμα.δοκιμή/Αρχική_σελίδα (a valid UTF-8 domain name).