13 ms·
> Can help you with Email Validation and API Calls It generates a nastily complex regular expression that is hopelessly wrong. Visible at https://www.youtube.c
by chrismorgan 5y ago
> Can help you with Email Validation and API Calls
It generates a nastily complex regular expression that is hopelessly wrong. Visible at https://www.youtube.com/watch?v=9Pw-Roo_duE&t=404 https://www.youtube.com/watch?v=9Pw-Roo_duE&t=404, here transcribed:
/^([\w-\.]+)@((\[[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.)|(([\w-]+\.)+))([a-zA-Z]{2,4}|[0-9]{1,3})(\]?)$/
For the local part, it requires [\w-\.]+, which excludes many valid characters like everyone’s favourite, +.
For the domain part, it tries to allow IPv4 addresses as well as normal domain labels (not IPv6 addresses, though), but it ends up tangling it up in a way that a human never would, allowing things like [12.34.56.com], [987.654.321.000, example.com] and example.123], while disallowing things like example.studio (the last label only allowing 2–4 letters) and IDN TLDs (which start with xn-- and must allow hyphen and numbers, not just [a-zA-Z]).
The author makes no comment on how hideously bad it is, which makes me suspect he didn’t notice, which… yeah, shows the problems of the whole thing.
- visarga 5y agoCopilot acts like a search engine, you search, you find, then you judge. It was never the case with search engines that you could just copy some code you found without verifying it. Also, it has the same copyright problems as if you used Google to find the code.
- objclxt 5y ago> Copilot acts like a search engine, you search, you find, then you judge. OK, but that’s not how GitHub position it: “Your AI pair programmer” “Skip the docs and searching for examples” They literally say it’s not a search engine!
- tsimionescu 5y ago> Also, it has the same copyright problems as if you used Google to find the code. No, it has the same copyright problems as if you Google and instead of getting links to sites that host code and licenses, you get just the code.
- philderbeast 5y agoIt's even more gray then that. Regardless of the licence, does the produced code even quality for copywrite protection, or does it fall under fair use? what licence, if any, is there for unique code generated by co-pilot etc etc etc. its a great big ball of who knows, however I expect that noting your only getting snippets you would be highly unlikly to get code that dosent fall under the fair use provisions, that said IANAL
- foepys 5y agoDoes GitHub Copilot tell me which license the code it suggested has? If not it's a huge difference to code search engines.
- foxpurple 5y agoCopilot is not a search engine. It synthesises code so it is unlicensed.
- roblabla 5y agoThe judge is still out on the licensing issue. And given it can, at least sometimes, output verbatim copy paste, including comments, of well-known GPL code[0], well, let's just say the issue is not clear-cut. [0]: https://news.ycombinator.com/item?id=27710287 https://news.ycombinator.com/item?id=27710287
- lumost 5y agoCopilot's api and suggestions could easily (and maybe was?) implemented as a SBQA style model. Using a search engine to find promising examples/context followed by a transformer model to synthesize the final output. Attribution would clearly be required in such a search derived model.
- dcow 5y agoSo if we classify AI as search.. and then claim fair use.. we can launder dirty viral code.. how could this go wrong? But really, if thispersondoesnotexist is just a really good per-pixel search against a corpus of human faces where each “page” of result pixels is organized in a grid presented as a new image with its own metadata.. I mean I guess Google really was an AI company all along.
- chrismorgan 5y agoNice theory. Won’t work out in practice, because this produces code that will run, and it’s AI, so it must be good, right? When you found code on the internet, it was presented in a context that let you make better judgement (e.g. on Stack Overflow this regular expression would have had a score of roughly −∞ and multiple highly-voted comments saying “do not use this, it’s catastrophically bad”), and where you have to put in more effort to plug it in and shuffle things around a bit as well. With Copilot, you get given ready-to-go code without any sanity checking at all. See even how, a few seconds later in the video, the author does test it out—but not thoroughly enough.
- visarga 5y agoI think Copilot should report the matching source URL to allow the user to visit the page and see the context and license. This move would also placate some copyright questions because it would be like searching StackOverflow or Github for inspiration. The problem of content attribution (exact and fuzzy match) has been studied before under the task of plagiarism detection for student essays. Funny thing is that a plagiarism detection Copilot would also disclose past cases of copyright violation and cause attribution disputes because code sitting unchecked in various repos would suddenly become visible.
- duskwuff 5y ago> I think Copilot should report the matching source URL That's the problem. The output of a GAN like Copilot usually can't be traced directly back to a single input.
- tyingq 5y agoSearch engines give you a link where you can (usually) see the code in context, who wrote it, when, license, etc. And often more, like who is using it where, how often it's updated, contact info, test suites, and so on.
- mikewarot 5y ago>Copilot acts like a search engine No, it doesn't. If my understanding of it is correct, it's an autoencoder, then a few more bits of AI. The MINST dataset is a collection of hand written digits used in many early machine learning classes. Usually they are used to train a classifier, which returns the correct digit given an image. They can also be used to train an autoencoder, which will take an image in, compress it down to far fewer channels, and put out an image that quite closely matches the original. Once you have an autoencoder, it is easer to input data, and train a neural network to do something with the compressed output. There is no way the autoencoder knows which samples were used to generate the resulting output, it's just optimized at compression. Thus, Copilot isn't search. You could take the entire corpus it was trained on, and log all the compressed outputs. You could then take a given output before the autoencoder expands it back out, tell which few source code fragments were closest, but there are no guarantees. TLDR; A far closer analogy: Copilot acts like a Comedian who has stolen a lot of jokes, and can't even remember where they came from.
- amelius 5y agoJust like a real copilot in a car or an airplane shouldn't be trusted? Perhaps they should choose a different name then.
- busymom0 5y ago> same copyright problems Is this true? From what I remember reading, the code was uniquely created (but I could be wrong). If that's the case, then does it tel you what license the generated code is under?
- hrithwikbha 5y agoHey Chris I am the author. When i made the video I didn't really notice the code part of regex, since I am really new to regex but in the conclusion part of my video I did mention that most of the code is not efficient Your comment was a great learning . Thank you
- lilyball 5y agoThis is exactly the problem. The regex issue isn’t that it’s not efficient, it’s that it’s wrong. Using this tool to generate code in a problem area you are not qualified to double-check and validate yourself is dangerous.
- omk 5y ago> Using this tool to generate code in a problem area you are not qualified to double-check and validate yourself is dangerous. I would like this message to be amplified as much as possible. Never write code you do not understand. I am excited about copilot, but also wary of the programming culture these tools will bring in. Businesses, especially body-shopping companies will want to deliver as much using tools in this category and end up shipping code with disastrous edge cases.
- ZiiS 5y agoIsn't "Code you don't understand" the definition of AI/ML?
- foxpurple 5y agoNo. You could use copilot to generate code you do understand and double check it before committing. It’s similar to just copying and pasting from stack overflow.
- tormeh 5y agoZing! But well, depends on the algorithm. Some aren't that complicated to understand, like linear regression. Others, like DNN are basically impossible. But with ML you're at least always testing the code you don't understand in the process of training the parameters. That's better than the minimum effort when using copilot code. And many will just make that minimum effort and release untested code they don't understand.
- alkonaut 5y agoThis has been discussed at length several times before and the answer to why you use a regex like that to validate emails is because you aren't trying to validate against a standard, but against a subset of email formats. You want a "normal" simple email. No "+" domains etc. Because it doesn't matter if you annoy the 0.01% of your users that would be negatively affected by that, it's better to have their simple/canonical emails. On the right side of the @ I agree completely, e.g. you must allow longer tlds. The regex is shit. But the same "simplification" thing would apply for IPv4: I'd probably want to have 4 groups of {0-9} even if a valid ipv4 address could be written in a lot more creative ways than that. The normal/simple/canonical way to write the address is a smaller scope than the set of allowed ways. The regex to parse any valid email and the regex to parse the info I want, (perhaps from the user subset I want!) can be very different. Edit: don't shoow the messenger - there is just zero chance you want a db that is more likely to contain user errors, has less valuable emails in it. "Enter your email" doesn't mean "Enter a string that can be considered valid according to the RFC"! No one cares whether "foo/baz=frob@example.com" is a valid email adress or not. In some cases you want RFC-compliant addresses, in which case it's a perfectly good idea to parse strictly to the spec. But in most cases you want a user identifier you know you can also contact with 100% certainty using some email SaaS. Or one you can cross reference to some other source. That's stricly a different purpose than parsing RFC compliant addresses. And allowing "foo@bar"@baz.com is just not a good idea.
- yreg 5y agoHow is it better to have "simple" emails?
- alkonaut 5y agoIt depends on the use case, but for some business use cases you e.g prefer access to the end users default/canonical inbox and not a specific one the user can use to filter etc. It’s also much less likely to cause downstream problems like distribution issues, be rejection by spam filters and so on. Annoying a tiny fraction of users or losing their business just isn’t a big enough issue to matter.
- avar 5y ago> It generates a nastily complex regular expression that is hopelessly wrong. > [...] > The author makes no comment on how hideously bad it is[...] I mean, it's coming up with a solution that's about as good as the average programmer who's going to validate E-Mail with regexes would, so as a crowd-sourced machine learning solution it's not too bad if you think about it. In other words, Having a co-pilot doesn't mean you're guaranteed to get Chuck Yeager.
- bamboozled 5y agoHere’s the difference imo, an average programmer with some experience would seek out well tested and used library to help them do something like this, not use sausage meet spat out of a cannon to validate email addresses.
- chrismorgan 5y agoI’d hope so, but then again, this is what their ML model spat out, which suggests people have written stuff like this, though hopefully not the wonkiness around the bit after the last dot. But on the brighter side, the material the user provided to Copilot in this case was pretty much “I want to implement email validation from scratch” rather than “I want to validate an email address”, which is where hopefully people would look more to existing libraries. And they’ll commonly already such libraries or functions in their code base, e.g. under Django you’d use… uh oh, searching found https://stackoverflow.com/q/3217682/ https://stackoverflow.com/q/3217682/ first which looks frighteningly familiar here in half of the errors it contains; but anyway, you should use https://docs.djangoproject.com/en/3.2/ref/validators/#emailvalidator https://docs.djangoproject.com/en/3.2/ref/validators/#emailv.... I suspect the Copilot approach as used will be unintentionally biased much more towards boilerplate and implementing things from scratch, rather than using libraries.
- yyyk 5y ago>about as good as the average programmer who's going to validate E-Mail with regexes would IMHO, the average programmer is not even aware of regexs, which is a problem, but here would lead the programmer to a simpler to read solution. You need to be at a very particular point (good enough to be proficient with regexs, but bad enough to use them for everything and bad enough to not test your regex), and I strongly doubt that's the average. P.S. The video question was to validate emails, not 'validate using regex'.
- cf499 5y ago> The author makes no comment on how hideously bad it is, which makes me suspect he didn’t notice, which… yeah, shows the problems of the whole thing. The author's comments on that "reverse" function are equally bad - https://youtu.be/9Pw-Roo_duE?t=171 https://youtu.be/9Pw-Roo_duE?t=171 It's described as "efficient" but it calls `len` on an unchanging list in 3 places.
- xoudini 5y agoWell, to be fair, the `len` operation on lists in Python is a constant time operation. What makes the example particularly bad is using a cast on the result of a floating-point division, rather than just using Python 3 integer division (i.e. the `//` operator). Copilot was clearly just spitting out Python 2 code here.
- cf499 5y agoI think the difference is that redundant re-calculation is a code smell in any language, at all times, whereas the float/int division issue requires knowledge of Python 2/3 syntax quirks.
- shadowgovt 5y agoIt's context-specific whether recalculation is a code smell. I've definitely gotten feedback from senior developers to just do len(x) repeatedly instead of "littering" (their words) the code with `xCount=len(x)` assignments (because they knew len was constant-time).
- cf499 5y agoSure, it makes sense if you find something like `len_x` or `x_count` significantly less readable than `len(x)` and if you don't have to worry about resources. Can't say I've been there though. I did a quick test of this `reverse` function (which probably shouldn't exist in the first place) and, unsurprisingly, it became ~30% faster when `len(arr)` was only called once.
- bluefirebrand 5y ago"Email address validation cannot be solved adequately with a regex" is something we need to start teaching somewhere. The RFC spec for emails is just way to permissive to make validating email addresses a winning move. I swear I wind up having a battle over email validation at every company I go to. There is inevitably a business person that says "Well what about this site, they do it" and then I have to dig into whatever that site is actually doing and likely find a valid email address that breaks their validation to prove it. And probably some junior dev (or senior who swears they did email validation flawlessly somewhere else and same story. I have to break their regex a bunch with valid emails they don't permit. And of course then it's an uphill battle convincing them that what I'm using are in fact valid email addresses. Or you get the "Well no one ever actually does weird things in their email addresses so it's fine" or "gmail doesn't let me register that address so you're wrong" Email is annoying.
- chrismorgan 5y agoI like the HTML spec’s version, used for <input type=email> validation: https://html.spec.whatwg.org/multipage/input.html#valid-e-mail-address https://html.spec.whatwg.org/multipage/input.html#valid-e-ma.... It allows all realistic inputs except for IP addresses (of dubious realism) and email address internationalisation (internationalised domain names are supported, but the local part is still currently stuck with ASCII, which is in keeping with its still-quite-limited support, though https://github.com/whatwg/html/issues/4562 https://github.com/whatwg/html/issues/4562 progresses in fits and starts).
- bluefirebrand 5y agoThat's a good approach honestly. I tend to just check for an @ and call it a day, validate it by emailing it and giving them a link to click if I need them to. It's really the only way to ensure it's a valid and active address. People just don't want to build it.
- jeffnappi 5y agoProper email validation: Contains '@' and a message sent to the address is delivered.