8 ms·
Free email address validation API for web forms
- pudo 13y agoThis may be very German of me, but the privacy implications of sending an email address to a third-party service before form submission appear murky to me. Also, we need to find a name for "give me some personal data in return for a minimal value add service" offerings.
- damoncali 13y agoValid concern, but in this case, it's Rackspace, not some fly by night.
- sneak 13y agoAs well as anyone who owns Mailgun's server or logs, and any of their employees who have access to it, and NSA (FISA), and DHS (PATRIOT NSLs), et c.
- damoncali 13y agoIt's just an email address and you're already giving it to an unknown entity (the app using the service), but point taken.
- claudius 13y agoIf it’s outside the EU, you would need permission to send the address there, and after PRISM, German data protection agencies decided to block all further requests based on the safe harbour agreements with the US.
- alexk 13y agoMailguner here: We don't store or use the email addresses that were submitted to the service.
- Terretta 13y agoYou said this here, and in your docs, it says: "No addresses submitted to the guardpost service are ever stored on any Rackspace servers. Nothing is persisted after the request is complete." However, the API is using GET. So unless you turned off logging, you are storing all tested emails in your web server logs which most places gzip up and archive indefinitely. In other words, I'd imagine these logs "persist" after the request is complete. ADDED: As an end user, you may or may not be considered that these logs will contain: 1. Your email address 2. The place you are making an account 3. The time you created the account And this information will be stored likely indefinitely (whatever the server log retention policy is). These logs also give mailgun and/or Rackspace a great resource for the membership rolls and adoption rate of any site adopting this service.
- paul_f 13y agoIf this is a problem for you, then don't use it. Steal some regex off a Stack Overflow page.
- Ryoku 13y agoI have to agree with pudo. Sending a user's email to a third party service has many privacy implications. I don't care if you share, not share, store, not store, or whatever; users are trusting you not to share their emails and you should comply.
- baudehlo 13y agoWould you be interested in something open source that does this? I would be willing to release https://www.emailitin.com/email_validator https://www.emailitin.com/email_validator to github. It does all the same checks as this except for the "valid user part for Yahoo.com" stuff.
- jontas 13y agoI would definitely appreciate an open source option, though to be honest I am more interested in the correction feature than I am in validation. I have been using the same regex[1] for years it gets the job done adequately--at least, I've never received any complaints from users or clients. I've used the regex on some relatively high profile sites--the kind where if someone was unable to signup with a valid email address we would've definitely heard complaints. 1. http://www.regular-expressions.info/email.html http://www.regular-expressions.info/email.html
- baudehlo 13y agoIt does the correction feature too. Try it :)
- LambdaBoy 13y agoI don't think anyone could say no to more open source.
- colanderman 13y agoPerhaps a good solution would be a JS library to validate and strip the local part, then ship the non-local part to Mailgun's servers for domain validation.
- Zikes 13y agoIf it doesn't actually send a validation email to the address, I don't see the value over a simple m/.+\@.+\..+/ They say up front that email validation is hard, and yes there are tons of edge cases and obscure tricks and rules and probably there's no guarantee that even they managed to get it right with this service, but ultimately the customer either puts in the correct address or they don't. If they're going to make a typo then it's far more likely that it would be a legal typo, and if they're going to intentionally enter a false address then it's likely it'd be a simple asdf@asdf.com. Edit: This was a bit of a knee-jerk reaction to what I at first saw as a redundant overcomplication, however as russjones points out below it has already proven its value in reducing bounce rates by a significant percentage. So, it might not fit my own limited use cases, but it certainly can't be ruled out entirely. Best of luck to the Mailgun team and I hope people smarter than I am can put this service to good use.
- npsimons 13y agoThis was my first response too; there's been a CLI tool (called vrfy) for a while that I've given up on using because it almost always says "send some mail, i'll try my best". That tends to be the ultimate test, which you can probably already do pretty quickly with a backend for a web form. EDIT: That being said, making a potential customer wait for thorough verification, on top of "spamming" them before they've (technically) bought from you would be unwise; I can see how this would have merit.
- swampthing 13y agoDid you read beyond #1 in the post?
- Zikes 13y agoYes, and none of those extra steps actually have a significant effect over simple regex checking to confirm the validity of the email, because there is no way to know what the user's correct email address actually is besides sending them a validation email. None. The domain may exist, the mailbox may exist, it may match all the rules for that particular mail exchange, it can pass this validator with flying colors, but you still have to send that validation email because I could just as easily have entered your email as my own, and that makes this a moot step.
- swampthing 13y agoThis is super cool - can't wait to implement it. Do you have any stats on how long calls usually take (mostly asking because of the DNS check)?
- russjones 13y agoI only have performance numbers for the parser right now, and on average it takes about 0.07 milliseconds to parse and validate an address. That's the pure RFC syntax part. The longest part of validation service is DNS checks, but once the DNS server warms up and starts caching lookups the roundtrip time is going to be the longest part of the request. We're still collecting reliable statistics and once we have them we can follow up with you.
- old-gregg 13y ago99+% of the time it's a couple of in-memory lookups plus a bit of CPU for syntax validation. The API endpoint is hosted in Chicago on a dedicated aggregation to DC networks (nice to be on Rackspace). Should be fast.
- knes 13y agoAn other alternative is using Mailcheck.js https://github.com/Kicksend/mailcheck https://github.com/Kicksend/mailcheck At least you are not sending email addresses to a 3rd party.
- sleepyhead 13y agoNot the same though. It just hints that you have a typo.
- Sephr 13y agoThe demo failed on Ian Goldberg's real email address, n@ai. TLDs work just like every other domain name in DNS.
- ceejayoz 13y agoIt may make business sense for something like an e-commerce site to reject this sort of technically valid but far more likely to be a typo e-mail address.
- baudehlo 13y agoWorks on my validator (which is also free, and does the same checks as this one with the exception of knowing the Yahoo trick): https://www.emailitin.com/email_validator https://www.emailitin.com/email_validator
- claudius 13y agoWhich also stumbles over a#\ b.\@c@[IPv6:2001:4dd0:fc8c::1] :)
- alexk 13y agoFirst bug report! Sure it should work, we'll fix that. Thanks for reporting!
- fintler 13y agoIt doesn't seem to work with my extremely common email address of: "me@example.com"@example.com Complete fail. /s Seriously tho, I like the design. It's refreshing to see someone avoiding another ugly perl-style regex.
- graylights 13y agoNot a valid email address since example.com doesn't have a valid mx record. It's a reserved domain.
- cleverjake 13y agojust a few example emails that are listed as invalid but aren't "email with spaces"@gmail.com "very.unusual.@.unusual.com"@mail.yahoo.com "very.(),:;<>[]\".VERY.\"very@\\ \"very\".unusual"@strange.example.com n@ai Another server-side alternative is isemail.info, which does validate all of the above.
- aarondf 13y agoGah, TIL right? I had no idea those crazy examples were legit.
- russjones 13y agoHi cleverjake, I'm the developer of this service at Mailgun. You can't actually register "email with spaces"@gmail.com or "very.unusual.@.unusual.com"@mail.yahoo.com with Google or Yahoo. Both addresses pass pure syntax checks but then the validator kills it when it notices that Google or Yahoo won't let you register addresses like that.
- Zikes 13y agoSo it would be valid if I entered "very.unusual.@.unusual.com"@mail.zikes.me, assuming I created that address on my mailserver?
- russjones 13y agoHi Zikes, that's actually a bug, thanks for finding it. I'll fix it, and yes, that will soon be marked as valid.
- cleverjake 13y agoHey russ, Yeah, I know you couldn't on gmail or yahoo, but I didn't know that you are using domain specific rules. Thats great!
- graylights 13y agoThey're only valid if you can own that email address and The only one that looks valid there is the last n@ai, since apparently ai actually has an mx record. The first two are legal by rfc but not allowed by the individual providers. The third is not legit by rfc 2606 because example.com is a reserved domain. If it was just a RFC validation then it should validate all of those (except maybe example.com). But they go beyond that: "Furthermore, the validator is ESP specific, so we can go way beyond valid syntax checks, bring in specific requirement for Gmail vs. Yahoo vs. Hotmail."
- harryzhang 13y agoEv and Taylor, this is awesome and free is icing on the cake. We'll be putting this to good use.
- graylights 13y ago"3. Mail Exchanger existence checks Again, due to the robustness principle, just because a host does not define MX records does not mean they can’t accept mail. Mail servers will often fall-back to A records to try and deliver mail. That’s why we go one step further than just a DNS query, we ping the Mail Exchanger to make sure that it actually exists." Plenty of boxes don't respond to pings (icmp). Can I assume you're doing a tcp scan on mail ports?
- old-gregg 13y ago> Can I assume you're doing a tcp scan on mail ports? Nope. We're an ESP ourselves. Mailgun has delivered (and accepted) many billions of emails since our launch a few years ago. We have a lot of data on 99.99% other ESPs in the world and the most common misspellings for them. We also have ESP-specific data on what kind of RFC-compliant email addresses they do not allow.
- lucb1e 13y agoUnfortunately this is broken just like all other attempts I've ever seen. If this is true: http://i.snag.gy/RSwiG.jpg http://i.snag.gy/RSwiG.jpg Then explain why the e-mail is arriving: http://i.snag.gy/gWPn5.jpg http://i.snag.gy/gWPn5.jpg Sure this is an extreme edge-case, but this was my second test. Who knows what else it rejects. Actually, why do we validate email addresses anyway? Whynot just try and send that validation email that you're going to send anyway? And on top of that, why would I ever trust a free third party service to check all my user's addresses?
- grey-area 13y agoA surprising number of people mistype their email addresses - if you can cut that down, it cuts down a lot on unnecessary bounced traffic, and the support costs of explaining to users that they've used the wrong email address to sign up with. I appreciate your point that email addresses are allowed to include all kinds of wacky characters, but if the check simply triggers a warning to double check the address, and doesn't block registration, I don't see the harm in doing it - it reduces a lot of unnecessary problems for ordinary users signing up and making mistakes who don't have addresses like the one above.
- alexk 13y agoHi lucb1e, thanks for finding this issues. It actually gets validated correctly by our parser (which will soon be open source) but hits a bug in our API. We are working on a fix right now.
- jodrellblank 13y agoUnfortunately this is broken just like all other attempts I've ever seen. Unfortunately you didn't read the blog like most other commentors posting about ridiculously unusable but otherwise valid addresses: """Our goal is not to make a perfect address validator that can validate every single address that has ever been created. Our goal is to build a realistic address validator for the types of addresses we see everyday."""
- peterwwillis 13y agoI'm going to come up with a free password validation API for web forms. Just call my API with your username and password and the service it's used for and i'll return 200 OK status if it's a secure password.
- jasonlotito 13y agoYou joke. Heck, I was going to build it as an April 1 hackathon project. Someone actually built it. I forget the name, but yeah. User accounts as a service.
- cabirum 13y agoehew46ujwhtaeg@w4tg.com is valid. A few more ideas: String Concatenation API, Number Addition API, String Length API...
- darkhorn 13y agoWhy just don't use <input type="email" name="email"> and then send it to validate?
- claudius 13y agoa#\ b.\@c@[IPv6:2001:4dd0:fc8c::1] also fails.
- grey-area 13y agoThere are a few concerns about privacy and response times using this approach, but you can get a very similar effect just by writing/hosting a little bit of js locally which consults a list of common errors, does a simple syntax check, and shows a warning beside the email entry box asking if the user is sure about the mail address they are submitting. That's enough to make users look hard for any mistake, and can be done in about 10 lines of js, automatically whenever the email field loses focus. I've found even a very simple script checking common errors/domains has a significant effect on typos and bounce rates, and it has the advantage of not sharing the email and being a lot quicker. If you're only ever warning the user to take a second look, you don't have to worry about false positives. I'm not sure that anything other than basic checks adds a lot of value, and I'd worry about sending off users' email addresses to an API on a third party website before they've even agreed to terms - I don't think most users would be happy to find out that was happening. It would be interesting to experiment with different levels of checks, and see which ones provide the most value though.
- lelf 13y agoвася@мгу.рф (valid by rfc6531)
- maddddddddddddd 13y ago"This is not intended to be a bulk mailing list cleaning service and we reserve the right to disable your account if we see it being used as such." the only thing different between "bulk mailing list cleaning" and "new account validation" is volume... if you don't want people to use your API, don't offer it.
- Narkov 13y ago"We know that gmail.com is a valid MX host while gmali.com is not." They've failed in their own example. gmali.com has valid MX records and accepts mail. Just because they don't accept billions of email per month, why should this service block any mail?
- adrr 13y agoBecause you'll probably have more people who accidentally type gmali.com instead of gmail.com then people who have an actual gmali.com email addresses.
- abirkill 13y agoWhen I enter fred@gmali.com on their validation demo, I get a 'Did you mean' message, but the icon is a warning indicator (yellow exclamation point), rather than the error indicator (red cross). When I enter fred@gmal.com, I get a 'Did you mean' message, with the red cross error icon. I presume this means that it's detecting that gmali.com is a valid domain and can receive e-mail, but for most people it's not what they actually meant, whereas gmal.com is both a probable typo and a domain that cannot receive e-mail, and therefore invalid. In other words, I think it's doing the right thing, in that it's detecting that fred@gmali.com is a valid address, but warning you that it's not the correct one. I think there is definitely a usability improvement though, as it's easy to miss the yellow warning icon, and assume that it's actually telling you that you can't use that address.
- lebek 13y agoJust for fun: as an AngularJS directive/validator https://github.com/lebek/angular-guardpost https://github.com/lebek/angular-guardpost
- andrewcooke 13y agoin general, this sounds good (particularly the big data approach - gmail.com v gmali.com based on stats). but i am concerned that they don't mention RFC 3696 when describing their grammar. it's all there (and implemented in - the now unsupported - lepl). http://www.faqs.org/rfcs/rfc3696.html http://www.faqs.org/rfcs/rfc3696.html http://www.acooke.org/lepl/rfc3696.html http://www.acooke.org/lepl/rfc3696.html
- alexk 13y agoFor the folks with privacy concerns: we are actually planning on open sourcing the entire service as well as our MIME handling library. So if you have privacy concerns, you'll be able to run it locally.
- dudus 13y agoGuys after weeks of works I think I found the best way to validate emails. It's the regex below. /@/ But most of the times it's just overkill and you shouldn't care. It's free and you don't need mailgun or anything else. You are welcome
- aytekin 13y agoThere is great wisdom in your joke. We have a web form builder service. (Jotform) We serve 3 million forms and process hundreds of thousands of submissions daily. Every form pretty much has an email question and most people enable validation feature. We first started with a very very smart and long email validation check that was going to be perfect. Every time users reported a case where our regular expression didn't envision we had to reduce it. During the years we had to change it so many times, I am pretty sure we have left with something like this: Does it have an @? Does it have a period? Great! You are validated!
- georgemcbay 13y ago"Does it have an @? Does it have a period? Great! You are validated!" Technically a period is not required, you can have an email address directly on a top-level domain. Though, of course, that is a rare enough use case that I'm sure anyone who has one has long since given up on expecting it to be validated correctly by most webforms.
- mattbarrie 13y agololz all your emailz are belong to us!!
- rcsorensen 13y agoThis is a beautiful thing you've put out. Thank you. The ASCII guardpost in the API docs is also pure gold.
- tn13 13y agoThey just want to get access to email address of people!
- DenisM 13y agoThis is great, thanks for your effort. Couple more things I'd love to see: 1. Parse a chunk of text and salvage any email addresses from it that you can find. Use case: my users upload spreadhseets with email of their other team members, but email field would often contain more than one email (separated by slash or space or coma or god knows what), or other stuff like Skype account etc. 2. Actual validation service. I'd pay for it at standard mailgun rates, it would be easier for me than rolling my own as I do now. Thanks again!
- russjones 13y agoCheck out the /parse endpoint for our API, it does exactly what you want: https://api.mailgun.net/v2/address https://api.mailgun.net/v2/address
- jlgaddis 13y agoThe best "email address validation" that ever existed was SMTP's VRFY verb (and EXPN was quite useful too). Unfortunately, the spammers killed that real quick.
- alexk 13y agoYep, we miss this command too
- jlgaddis 13y agoThis looks pretty awesome and my first thought is it would be sweet if it was integrated into common webmail software. I work for an ISP and we, of course, provide e-mail access via webmail. Right this moment, I can see dozens and dozens of e-mail messages queued up on our outbound relays that will never be delivered because the user typo'd the recipient's e-mail address. An amazingly high number of messages bounced back to our users (the original senders) are due to typo's like this. Some people, despite not being "techies" can skim over a bounce message and realize they misspelled "live.com" and will resend. Others, well, they call support wanting to know why they suddenly can't e-mail Aunt Sally.
- Sektor 13y agoAs long as it accepts the + symbol in the Gmail address I'll be entering I'm happy for sites to use whatever they want.
- wvh 13y agoIt doesn't seem to accept IDN addresses, though it's able to resolve the punycode equivalent just fine. I was just implementing email and name validation checks for a project myself. Luckily email addresses can at least be validated by a confirmation email; it's the real name field I have no clue what to do with. It's funny that after all these years, we still don't seem to have cracked these basic problems.
- kfury 13y agoOh the edge cases will be so angry. There are a very few engineers at Google who have Gmail account names shorter than 6 characters. It's not the norm, but they exist. Their addresses can't be validated. I'm sure there are special cases all over the place. It would be nice if Mailgun differentiated between 'this address is just malformed' and 'from what I know of [ISP], this address oughtn't exist'.
- blantonl 13y agoI concur. You've got to wonder whether or not the trade off between denying the edge-cases outweighs the benefits of "better" validation.
- alexk 13y agoWell, you are not forced to reject the email, you can just stick to the unobtrusive spellchecker, that just suggests in case if something suspicious. So the end decision is up to your app.
- kfury 13y agoThe spellchecker doesn't actually kick in all that often. The better solution is to give flags back differentiating between 'unlikely' and 'impossible' email addresses.
- nandemo 13y ago> Formal grammars (and specifically in our case a context free grammar) are a set of rules that define the structure of a string. Note that a regular expression is a formal grammar too.
- jitnut 13y agoThis is cool! One thing- as a mailgun user, can i use it to check the valid email addresses when one of my client sends an email to his customers? That would save some email credit him and reduce bounce rate.
- hk__2 13y agoIt rejects addresses that contain a @ in the local part, even if these are valid ones: "foo@"@…, "me@google.com"@google.com
- hopeless 13y agoTo all those that think this is no better than a regex: Yesterday a company invited 7 new users to their account using their email addresses. 3 of those addresses had typos in the domain names which this service would have caught. As it was, this error was only discovered when the service tried to send invitation emails to the new users and that's not a great UX. Validation emails, particularly those with a confirmation link, are a horrible horrible solution. They interrupt the user's process flow, taking them away from their web browser, possible delaying the process, and you'll also get users searching through their emails and clicking that link just to access their account (yep, really). I think I'm going to implement something like this Mailgun service plus sending a welcome email (with no confirmation link). If the welcome email bounces then I can handle that case but it should happen less often with the Mailgun live-validation.
- nmridul 13y agoHere an easier option for those concerned with privacy of their users. 1) Validate the email address first on your side (using regex). 2) Then send the domain name part to the service for validating and correction. Maybe append a fake username before the @. So if a user enters someone@yahoo.cm , I validate it first then send someotherperson@yahoo.cm to the the mailgun. Now your real user is protected. Now you don't send them the real user name but get most of the benefits.
- jonursenbach 13y agoAny plans on making this an actual library instead of an API?