18 ms·
Some useful regular expressions for programmers
- skered 6y ago~"Now we have two problems."
- pkroll 6y agoThe person who originally said 'Some people, when confronted with a problem, think "I know, I'll use regular expressions." Now they have two problems.' is Jamie Zawinski. And it's an amusing quote, but, that doesn't mean it's generally correct. In the original context, this was more about Zawinski's utter hatred for Perl. http://regex.info/blog/2006-09-15/247 http://regex.info/blog/2006-09-15/247
- asicsp 6y agoNice examples. Feedback for the author if they see this thread: 1) \S can be used instead of [^\s]. I haven't come across a regex flavor that supports \s but not \S. GNU grep supports \s and \S but doesn't support using them inside character classes. 2) For the first example, it is a weird mix of formatting for the code snippets. Rest of the examples are properly code formatted and specified without single quotes around the expression. 3) The fourth example uses variable length lookbehind, which isn't supported by Perl, Python (unless you use regex module), Ruby, etc. \K can be used instead if supported. Also, `<` at the start of the regex is a typo
- bmn__ 6y ago> variable length lookbehind, which isn't supported by Perl It is supported: https://perldoc.perl.org/perl5300delta#Core-Enhancements https://perldoc.perl.org/perl5300delta#Core-Enhancements Work-around for all previous versions: http://www.drregex.com/2019/02/variable-length-lookbehinds-actually.html http://www.drregex.com/2019/02/variable-length-lookbehinds-a...
- asicsp 6y agoGood to know that support is being worked upon. Seems limited as of now, this is what I get on v5.32: $ echo 'asb34 asgfdhgree56 jdf42' | perl -lne 'print join "\n", /(?<=\ba[a-z]*)\d+/g' Lookbehind longer than 255 not implemented in regex m/(?<=\ba[a-z]*)\d+/ at -e line 1.
- BiteCode_dev 6y ago> It is often best to avoid non-ASCII characters in source code. Indeed, in some cases, there is no standard way to tell the compiler about your character encoding, so non-ASCII might trigger problems. To check all non-ASCII characters, you may do [^\x00-\x7F]. Depends of the language. In Python 3, files are expected to be utf8 by default, and you can change that by adding a "# coding: <charset>" header. In fact, it's one of the reasons it was a breaking release in the first place, and being able to put non-ASCII characters in strings and comments in my source code are a huge plus. I mentioning this because in another article, the author said being used to code in "C++, C, Go, Java, JavaScript, Python, R, Swift, Rust, C#" so I find it kinda weird.
- kazinator 6y agoI wouldn't say that everything in \x00 to \x1f is OK in source code.
- chrismorgan 6y agoEh, I reckon they’re fine. In shell scripts especially (and sometimes elsewhere), if I want an ANSI escape code, I’ll type the actual bytes I want, because it’s more reliable and easier to type <C-V><Esc> and see ^[ in Vim than it is to worry about whether I’m using \u001b, \u{1b}, \x1b, \033 or something else, and whether I have '', "", $'', $"", printf, echo -e, &c. &c. Speaking generally, if you use the actual control characters, the code will either fail to run (parse or compile error) or work, whereas encoding your control codes with escape sequences is more hit and miss. I’ve seen quite a few others using control characters in this way too. Truth to tell, I’m quite liberal in using all of Unicode in my source code. About the only places I wouldn’t use the actual characters are when (a) the environment requires me to escape the thing; (b) the numeric values of the characters are more meaningful than the characaters themselves; and (c) when dealing with combining characters and modifiers, because otherwise they’ll do things like combine with the opening ' from the string/character literal.
- kazinator 6y agoI mean, in non-strawman examples of programming languages. The code will work, but how will it edit outside of your personal environment? Or print? If you quote it in a PDF paper, will it be copy-pastable? Basically I would worry about all that sort of stuff. If I have echo '^L' to clear the screen, and that program is sent to a line printer, will I get: echo ' [ paper ejected ] ' If you so much as "cat" your code to a terminal, all the codes get interpreted.
- web007 6y agoMy favorite regex is /[ -~]*/, space-dash-tilde. That represents approximately A-Za-z0-9 plus punctuation and space. It's useful for something like `tr -d` to remove all common characters and leave all of the "oddities" so you can differentiate "" from “” or ... from … in things like markdown or source code.
- O_H_E 6y agoOh! That's nice to know.
- zvr 6y agoAdd a tab character in there (before the space) and you're even better!
- chrismorgan 6y agoExplanation: [ -~] is matching the range of characters† from space (32) to tilde (126), which is the full range of printable ASCII characters. (0–31 are various control characters, and 127 is one last control character, ␡.) † “Characters” is a deliberately vague term here. Different regular expression engines match different types of things, typically supporting one or two modes. The three most common things to match are bytes (the traditional default, normally encoding-unaware but potentially known to be ASCII or UTF-8), UTF-16 code units (JavaScript’s default, and UTF-16 was an all-round terrible idea that ruined Unicode for everyone else while achieving nothing useful), and Unicode scalar values (which is typically called “Unicode mode” and often enabled by a /u flag).
- deleted 6y ago[deleted]
- nwallin 6y agoFour of those were to catch formatting errors. IMHO it's best to just use an automatic code formatter. clang-format is extremely good. When I write code for personal projects I don't even bother trying to format it correctly. The only whitespace button I use is the spacebar. tab is mapped to the function that reformats the entire document according to the formatting rules I've configured. At work, whenever I apply a change in a function I will autoformat that function whenever. I can't just format the entire file, but formatting errors are rife in the codebase I work on. goto fail; is real.
- wilsonrocks 6y agoBinding auto format to the tab key has blown my mind!
- llarsson 6y agoAlternatively, do it on save. :) gg=G in vim is fast enough to type for me, though.
- tdfirth 6y agoSecond this. I'm pretty happy writing code in a plain old text editor much of the time, without most IDE features. One thing I just can't live without, however, is a code formatter. Formatting code by hand is tedious and repetitive and I have no patience for it.
- nuriaion 6y agoWe made it part of our ci to test if all files are formated with the formater and i bound the formater to the save operation. So i can just write, save the file and everything is nice :)
- williamdclt 6y agoDefinitely. Install a code formatter, make it part of your linting process so that it fails your CI, run it on save in your editor, never have boring conversations about formatting ever again with your team.
- 6y ago
- hnlmorg 6y agoComing from a Perl background I do love regex but this is just the wrong use for it. 1. Those expressions fall down on a number of edge cases (eg heredocs / string literals) 2. It is actually perfectly reasonable to have non-ASCII inside source code because these days it's increasingly common for compilers to assume source it UTF8 and, frankly, English isn't the first language for most of the speaking world so it's rude of us to assume all comments and quoted strings should be in English. 3. There's already multiple code formatters for nearly every language out there and they take tokenised versions of your source (ie will take into account stuff like the string literals I mentioned earlier). You're far better off picking whatever code formatter is considered (de facto) standard for your language and using that. They come with the additional benefit that they don't just highlight formatting faux pas but they can also enforce style guidelines.
- linkdd 6y agoMy favorite regex is the E-Mail Regex[1]. Works 99.99% of the time, is unreadable/unmaintainable, makes you understand that the only true way to verify an e-mail is to send an e-mail to that address. [1] - https://emailregex.com/ https://emailregex.com/ PS: Gotta love the Perl / Ruby version
- madsohm 6y agoThe only way you should ever "validate" an email address with a regex is like this: /@/
- oneeyedpigeon 6y agoSurely that should be /.+@.+/
- linkdd 6y agoYour regex would validate: this-is-not-a-valid-address@ @this-is-not-valid-either @
- wongarsu 6y agoBut allowing too much is better than allowing too little, as usually you have to send an actual email to verify ownership anyways. Any regex more complex than /.+@.+/ fails some valid email address
- linkdd 6y agoWhich was the point of my first message.
- PeterWhittaker 6y agoSurely that should be /^[^@]+@[^@]+$/ ?
- 6y ago
- morelisp 6y agoA personal common favorite: "(?:[^\\"]|\\.)*"
- eyelidlessness 6y agoDisclaimer: none of what follows is criticism of you, of your contribution, or even of regex. I’m actually tickled by such a terse comment giving me so much to think through. - - - This is my fourth attempt at understanding the intent of this pattern. I’m quite comfortable with regex, so much so I listed it on my resume skills list for years as sort of a glib joke. I think I’ve been overthinking this one. For my benefit I’m gonna describe what I think this matches and ask you to confirm. For others I hope the description helps. This appears to be intended to match a balanced double quote, where any characters between may be anything besides a backslash or double quote, or any character preceded by a backslash. The content between the bounding quotes isn’t captured, which is to say parentheses usually designate a capturing binding like $1 or \1 depending on language/context, but the ?: tells the regex engine you don’t need/want that binding. If I’m understanding all of that correctly, it’s probably very close to what you intended but definitely highlights two of the biggest problems with regex: 1. Intent is easily lost on read. This is trite to say, but I’m pointing it out as someone who’s played regex golf for fun and still spent over an hour getting confident enough to describe what this short pattern appears to do. 2. Balanced pairs allowing an escaped terminator are not “regular language”. It’s possible (and I think probable but I’m not at a device capable to confirm) that this pattern is safe because the balanced pairs are single-character and there’s only one possible escape. But the fact that I can’t say so with confidence is something that would make me cautious using it. All of that said, if I were writing this. I’d try to make it both more clear in intent and a little more predictably resilient. 1. I’ll include slash delimiters to remove ambiguity. /"(?:[^\\"]|\\.)*"/ 2. I’ll express the escape exception first: /"(?:\\.|[^\\"])*"/ 3. I’ll remove the need to check unescaped characters by terminating the match on the first unescaped quote (using ? after a wildcard is non-greedy and stops at the first match; the default is to match to the last): /"(?:\\.|.)*?"/ I think this makes me feel a little more confident with the escaping, but I’m still iffy without testing it. But it does make for a pretty cool emoticon! Final thoughts: - Almost everywhere you could use regex will not match any kind of line breaks. That might be desirable but depends on what you’re trying to do. - Even with the abundance of caution I expressed, this is decidedly unsafe for processing user input. For example I’m certain this would be an XSS vulnerability.
- The_rationalist 6y agoFor the one that can't stand the cognitive overhead of regex like me, verbal regex is a major improvement: https://github.com/VerbalExpressions/JavaVerbalExpressions https://github.com/VerbalExpressions/JavaVerbalExpressions
- morelisp 6y agoI can't imagine finding VerbalExpression regex = regex() .find("a") .capture().find("b").anything().endCapture().then("cd").build(); To involve less "cognitive overhead" than /a(b.*)cd/ For example, what is "anything"? (In this case, "anything" can be nothing.) Where does the capture group start/end? (In normally regexp syntax even naive brace matching will usually tell you.)
- dragonwriter 6y agoI don’t see how that’s less cognitive overhead. It uses words, but not only are some of the choices individually dubious for what they do (using domain specific symbols isn’t higher cognitive load than using common words with no obvious domain specific meanings), it breaks a cardinal rule of naming in an API by using different parts of speech to name things of the same kind, mixing verbs (“find”) and nouns (“anything”). It’s way more verbose, but not, in any way I can tell, lower cognitive load.
- morelisp 6y agoEven as structured regular expression languages go it's quite a poor example since it doesn't capture any of the, uh, structure, of the regular expression. In the end it still has all the downsides of the linear string syntax (and none of the upsides - it's not composable at all either, for example), isn't really clearer, and it'll be immensely slower.
- kyoob 6y agoNice! Added a few of these to my Anki deck.
- Minor49er 6y agoAt my former job, the dev team wasn't allowed to use a formatter, so people would often commit code that would be off by a space or three. Similar to the post, I got into the habit of detecting these with this: ^( {4})\* {1,3}\S There were cases where the backend would generate a stack trace, but HTML-format it in a response that was not expecting HTML. I could select and delete them with this, which would make the results instantly more readable: <[^>]+>
- dboreham 6y agoWasn't allowed to??? A giant hand smote your head if you typed the code format command?
- Minor49er 6y agoMetaphorically, yes. I mostly built off of existing code while I was there. Frequently (and not just for myself), I would see code getting held up in review because the more tenured developers who looked at it would say that the formatting was beyond the scope of the changes being made and would refuse to approve anything until all of the whitespace was reverted. This same class of devs would also commit new changes with more cases of these annoyances, such as trailing whitespace and mixed tabs/spaces, but were able to merge their unreviewed stuff in without management batting an eye.
- eyelidlessness 6y agoThis just gave me an idea. I’ve been on teams like this. I’ve also been on teams at the other extreme, where formatting nit picking took up the vast majority of review. The former is obviously painful because poorly formatted code is a constant mental burden. But it’s also true that unrelated formatting changes can present a similar mental burden and make meaningful changes harder to identify. The latter can detract from more meaningful review. But prioritizing better formatting consistently keeps a codebase more manageable over time. This is somewhere tooling can help, more than it already does. Automated linting/formatting is great. Depending when it’s applied, and how effective its automation, it can potentially defer the burden til review time. But there’s still a lot of room for improvement there. Many diff tools present an “ignore whitespace” option. I would like to see this taken two steps further: 1. Better language syntax awareness could go beyond whitespace as formatting-specific “noise”. I know there’s work in this direction and it’s great too. Besides making it more available and complete... 2. “Ignore” is probably not the best option. Ideally tooling can also identify these two sets of changes so they can be both viewed, and more importantly merged separately. The problem right now is that formatting is noise pretty much everywhere one might interact with a project. Making it a distinct set of changes at the diff and history level would help significantly.