7 ms·

Most of the time I mention the topic of regular expressions to other developers, I usually hear self-critical commentary like "oh, I'm terrible at regex", and r

Most of the time I mention the topic of regular expressions to other developers, I usually hear self-critical commentary like "oh, I'm terrible at regex", and rarely anyone who loves them. I think they're great though, if you take the time to understand them. They're something like a Swiss Army knife for programming.

BenjiWiebe 9y ago
I love them and I think I'm great with them. (Though I'm probably not as great at them as a bunch of you reading this comment)
jstarfish 9y ago
They're definitely useful, and I can cobble them together to get lots of otherwise tedious and complex parsing tasks done, but when I come back to them a week later I have no idea what the hell the pile of wingding vomit I wrote was supposed to do. I find myself writing simpler ones and tying them together with app code just for sanity's sake.
- testestx 9y ago
  > I find myself writing simpler ones and tying them together with app code just for sanity's sake. Or just use PEG, parser combinators, or other more readable parsing abstractions
- TeMPOraL 9y ago
  Some regex implementations allow for comments in the string; if your does not, you can probably make it work with concatenation, like: String pattern = "^https+" // match the protocol at the beginning + "([a-zA-Z])+" // match the machine name + ... Honestly, I use regular expressions because, even in such format expanded with comments, I haven't seen anything more readable after you get used to regex operators. I guess the closest would be the alternative format in CL-PPCRE. For instance: CL-USER> (cl-ppcre:parse-string "\\b\\d{1,3}\\.\\d{1,3}\\.\\d{1,3}\\.\\d{1,3}\\b") (:SEQUENCE :WORD-BOUNDARY (:GREEDY-REPETITION 1 3 :DIGIT-CLASS) #\. (:GREEDY-REPETITION 1 3 :DIGIT-CLASS) #\. (:GREEDY-REPETITION 1 3 :DIGIT-CLASS) #\. (:GREEDY-REPETITION 1 3 :DIGIT-CLASS) :WORD-BOUNDARY) But then, any such form can get mouthful: CL-USER> (cl-ppcre:parse-string "((\\b[0-9]+)?\\.)?\\b[0-9]+([eE][-+]?[0-9]+)?\\b") (:SEQUENCE (:GREEDY-REPETITION 0 1 (:REGISTER (:SEQUENCE (:GREEDY-REPETITION 0 1 (:REGISTER (:SEQUENCE :WORD-BOUNDARY (:GREEDY-REPETITION 1 NIL (:CHAR-CLASS (:RANGE #\0 #\9)))))) #\.))) :WORD-BOUNDARY (:GREEDY-REPETITION 1 NIL (:CHAR-CLASS (:RANGE #\0 #\9))) (:GREEDY-REPETITION 0 1 (:REGISTER (:SEQUENCE (:CHAR-CLASS #\e #\E) (:GREEDY-REPETITION 0 1 (:CHAR-CLASS #\- #\+)) (:GREEDY-REPETITION 1 NIL (:CHAR-CLASS (:RANGE #\0 #\9)))))) :WORD-BOUNDARY)
  - arkh 9y ago
    If you're using PCRE you should also make use of named patterns. It makes the expression easier to understand as you can reuse parts of it (a little like functions) and the matched patterns can be then used in your language with their name instead of their position. Decoupling the usage from the regexp so it is more robust. http://www.rexegg.com/regex-capture.html#namedgroups http://www.rexegg.com/regex-capture.html#namedgroups
    - reificator 9y ago
      That would be nice to have when working in Javascript, making code a lot more readable and easy to update. Alas, we don't get such luxuries as named groups or static typing... Woe is me.
- dash2 9y ago
  I wonder if Perl 6 regexes and grammars might show a way forward for more readable pattern matching.
  - b2gills 9y ago
    An example would be helpful so here is one https://github.com/moritz/json/blob/master/lib/JSON/Tiny/Grammar.pm https://github.com/moritz/json/blob/master/lib/JSON/Tiny/Gra...
Retra 9y ago
A programmer saying they are terrible at regex is like a mathematician saying they are terrible at algebra.
- m3kw9 9y ago
  Mobile programmers doesn’t need regex nearly as much as serverside
  - Retra 9y ago
    JS programmers don't need to know how a C pointer works, but they're not doing themselves any favors by being ignorant of it. It's very basic basic background knowledge.
    - m3kw9 9y ago
      I meant to say they do not need to be great beyond basics like using wild cards, and simple Inclusion and exclusion syntax. That’s probably more than 50% of time all you need on that side of he world
      - Retra 9y ago
        Yeah, that's true. If you understand the quantifiers and matching groups, you basically understand regex. That's all I'm referring to. I'm not saying everyone has to be an expert on implementing them.
- stochastic_monk 9y ago
  I’m not terrible at it, but while useful in general, I rarely use regular expressions in the core programs I develop. I think the automata theory behind them is more important to know than proficiency with specific regular expression implementations.
- roel_v 9y ago
  Regex is useful when you do lots of string processing, like in webdev. Outside of that, I've found uses to be very limited - certainly not worth the upfront time investment. (I mean, sure one can cobble together something that mostly works with a regex testing tool, but you need to either take a college automata course or work through the Friedl in detail to get a basic level of proficiency).
qiqitori 9y ago
I think everyone who doesn't know regex should make learning regex a priority. (However, I find that lookahead and lookbehind in particular do not tend to come in handy very often. So maybe just make a mental note that this exists and then look it up when you need it.) Just learn the basics and maybe take a very quick look at the theory, finite automata (maybe the name puts people off, but its just a couple of circles connected to other circles with a bunch of characters written on the connecting lines. I'm pretty sure you could explain it all in a few sentences a few examples). You'll get an intuitive feeling for what you can and can't do with regular expressions. You don't even have to be able to code to make use of regular expressions. You can use regular expressions when searching and replacing in editors (even slightly barebones editors like gedit or kate). You can transform input data from almost any format into any other format using nothing but your editor and a series of replace statements. (No computations though.) I think they should teach regex in high school. Many people working in non-IT office jobs could benefit from knowing regex, and I think it's really quick to learn this. (Now if only Excel's/Word's search/replace supported regex...)
- Someone1234 9y ago
  Which version of RegEx? I've "learned" RegEx two or three times and then switched language/platform and had everything I previously learned no longer work reliably. You might think I am just talking about Microsoft's quirky implementation but even in the Linux-sphere it isn't consistent see: http://www.greenend.org.uk/rjk/tech/regexp.html http://www.greenend.org.uk/rjk/tech/regexp.html You take a complex format string which was design to use the fewest characters instead of with clarity in mind, you then have every major application and library diverge on basic support and spec for features, and then you have all of them hack on support for UNICODE in their own unique way. Regular Expressions likely won't ever die, but I for one would happily switch to an alternative with better readability, UNICODE support from day zero, and fewer niche features to keep things uniform. I'm tired of re-learning RegEx only to have everything I've learned either be forgot or not work the second I app switch.
  - kbenson 9y ago
    For the most part, it's just a matter of knowing if you're using POSIX Basic Regular Expressions (BRE), POSIX Extended Regular Expressions (ERE), or Perl regular expressions. Learn those, or at least the main differences between them, and the vast majority of the regular expression engines in software you use will become more recognizable.
keithnz 9y ago
I love regex. Not just for doing pattern matching in code, but for searching and data transformation in editors and tools. But yes, lots of people do seem to do the "I suck at regex". Even when I notice people do crazy long winded transformations by hand which I then do within seconds. Still doesn't seem enough motivation for them to learn them properly.
zaarn 9y ago
Regex's are awesome as a swiss army knife. Though in my experience, outside some edge cases where the format never changes (like matching a domain name in a URL) a regex is hell to maintain when you come back 3 years later. There is also always the fun of people trying (and failing) to use regex in emails.
- ulucs 9y ago
  .+@.+ seems to be the only one without too many false-negatives
  - zaarn 9y ago
    Well the best solution is to check if .+@(.+) matches and then try to lookup what the capture group returned via your DNS resolver. If it has an MX record (or CNAME to something with MX), then deliver the mail to there. If you can't resole the domain part, return an error.
reificator 9y ago
Regex is great for very simple text processing, and I tend to do a lot of that. The most important parts to get comfortable with are: * Capture groups and alternation * Character sets * Anchors (start and end of line) * Common escape sequences (digit, word, whitespace) * Repeat (any, one or more, n-m) * Common flags (global, multi-line, case-insensitive) If you need more than that, it's time to start evaluating other tools IMO. A lookahead here and there is okay, but I'd avoid them if possible. The best way I found to learn regex was to take a set of inputs that I wanted to match, (and a set that I didn't) and play around on https://regex101.com/ https://regex101.com/ until I got a pattern that did what I wanted. You'll very quickly start to learn the above bulletpoints, and before long you'll be able to write patterns without any reference. If you find that your regexes are getting too large or unwieldy or difficult to understand (despite knowing the above bulletpoints) then you probably need a parser or some other more suitable tool.
DaveChild 9y ago
When I was learning my way around regexes, I found a cheat sheet helped. I made my own - you can download it free from here: https://www.cheatography.com/davechild/cheat-sheets/regular-expressions/ https://www.cheatography.com/davechild/cheat-sheets/regular-...
pmarreck 9y ago
I absolutely unabashedly love Regex. It's like solving a puzzle that ends up eliminating work (by doing said work) extremely concisely. What better kind of puzzle is there?
zombieprocesses 9y ago
> I think they're great though, if you take the time to understand them. Regular expression is useful for sure. What is terrible is that every language, shell and platform has different "styles and implementation". On windows, cmd, powershell, C#, sql server, etc all have their own styles. It's similar enough and at the same time different enough to drive you insane. Throw in linux with their shells, vi(m), perl, etc all using their own variants. But the biggest problem is that regex is prone to "set it and forget it" issue. It's something we use once in a while and forget. Was it brackets or parentheses or braces for defining character ranges? Does . or + signify one or more? And ever try deciphering someone's undocumented multiline regex? Fun times.

BenjiWiebe 9y ago

I love them and I think I'm great with them. (Though I'm probably not as great at them as a bunch of you reading this comment)

jstarfish 9y ago

They're definitely useful, and I can cobble them together to get lots of otherwise tedious and complex parsing tasks done, but when I come back to them a week later I have no idea what the hell the pile of wingding vomit I wrote was supposed to do. I find myself writing simpler ones and tying them together with app code just for sanity's sake.

testestx 9y ago
> I find myself writing simpler ones and tying them together with app code just for sanity's sake. Or just use PEG, parser combinators, or other more readable parsing abstractions
TeMPOraL 9y ago
Some regex implementations allow for comments in the string; if your does not, you can probably make it work with concatenation, like: String pattern = "^https+" // match the protocol at the beginning + "([a-zA-Z])+" // match the machine name + ... Honestly, I use regular expressions because, even in such format expanded with comments, I haven't seen anything more readable after you get used to regex operators. I guess the closest would be the alternative format in CL-PPCRE. For instance: CL-USER> (cl-ppcre:parse-string "\\b\\d{1,3}\\.\\d{1,3}\\.\\d{1,3}\\.\\d{1,3}\\b") (:SEQUENCE :WORD-BOUNDARY (:GREEDY-REPETITION 1 3 :DIGIT-CLASS) #\. (:GREEDY-REPETITION 1 3 :DIGIT-CLASS) #\. (:GREEDY-REPETITION 1 3 :DIGIT-CLASS) #\. (:GREEDY-REPETITION 1 3 :DIGIT-CLASS) :WORD-BOUNDARY) But then, any such form can get mouthful: CL-USER> (cl-ppcre:parse-string "((\\b[0-9]+)?\\.)?\\b[0-9]+([eE][-+]?[0-9]+)?\\b") (:SEQUENCE (:GREEDY-REPETITION 0 1 (:REGISTER (:SEQUENCE (:GREEDY-REPETITION 0 1 (:REGISTER (:SEQUENCE :WORD-BOUNDARY (:GREEDY-REPETITION 1 NIL (:CHAR-CLASS (:RANGE #\0 #\9)))))) #\.))) :WORD-BOUNDARY (:GREEDY-REPETITION 1 NIL (:CHAR-CLASS (:RANGE #\0 #\9))) (:GREEDY-REPETITION 0 1 (:REGISTER (:SEQUENCE (:CHAR-CLASS #\e #\E) (:GREEDY-REPETITION 0 1 (:CHAR-CLASS #\- #\+)) (:GREEDY-REPETITION 1 NIL (:CHAR-CLASS (:RANGE #\0 #\9)))))) :WORD-BOUNDARY)
- arkh 9y ago
  If you're using PCRE you should also make use of named patterns. It makes the expression easier to understand as you can reuse parts of it (a little like functions) and the matched patterns can be then used in your language with their name instead of their position. Decoupling the usage from the regexp so it is more robust. http://www.rexegg.com/regex-capture.html#namedgroups http://www.rexegg.com/regex-capture.html#namedgroups
  - reificator 9y ago
    That would be nice to have when working in Javascript, making code a lot more readable and easy to update. Alas, we don't get such luxuries as named groups or static typing... Woe is me.
dash2 9y ago
I wonder if Perl 6 regexes and grammars might show a way forward for more readable pattern matching.
- b2gills 9y ago
  An example would be helpful so here is one https://github.com/moritz/json/blob/master/lib/JSON/Tiny/Grammar.pm https://github.com/moritz/json/blob/master/lib/JSON/Tiny/Gra...

Retra 9y ago

A programmer saying they are terrible at regex is like a mathematician saying they are terrible at algebra.

m3kw9 9y ago
Mobile programmers doesn’t need regex nearly as much as serverside
- Retra 9y ago
  JS programmers don't need to know how a C pointer works, but they're not doing themselves any favors by being ignorant of it. It's very basic basic background knowledge.
  - m3kw9 9y ago
    I meant to say they do not need to be great beyond basics like using wild cards, and simple Inclusion and exclusion syntax. That’s probably more than 50% of time all you need on that side of he world
    - Retra 9y ago
      Yeah, that's true. If you understand the quantifiers and matching groups, you basically understand regex. That's all I'm referring to. I'm not saying everyone has to be an expert on implementing them.
stochastic_monk 9y ago
I’m not terrible at it, but while useful in general, I rarely use regular expressions in the core programs I develop. I think the automata theory behind them is more important to know than proficiency with specific regular expression implementations.
roel_v 9y ago
Regex is useful when you do lots of string processing, like in webdev. Outside of that, I've found uses to be very limited - certainly not worth the upfront time investment. (I mean, sure one can cobble together something that mostly works with a regex testing tool, but you need to either take a college automata course or work through the Friedl in detail to get a basic level of proficiency).

qiqitori 9y ago

I think everyone who doesn't know regex should make learning regex a priority. (However, I find that lookahead and lookbehind in particular do not tend to come in handy very often. So maybe just make a mental note that this exists and then look it up when you need it.) Just learn the basics and maybe take a very quick look at the theory, finite automata (maybe the name puts people off, but its just a couple of circles connected to other circles with a bunch of characters written on the connecting lines. I'm pretty sure you could explain it all in a few sentences a few examples). You'll get an intuitive feeling for what you can and can't do with regular expressions. You don't even have to be able to code to make use of regular expressions. You can use regular expressions when searching and replacing in editors (even slightly barebones editors like gedit or kate). You can transform input data from almost any format into any other format using nothing but your editor and a series of replace statements. (No computations though.) I think they should teach regex in high school. Many people working in non-IT office jobs could benefit from knowing regex, and I think it's really quick to learn this. (Now if only Excel's/Word's search/replace supported regex...)

Someone1234 9y ago
Which version of RegEx? I've "learned" RegEx two or three times and then switched language/platform and had everything I previously learned no longer work reliably. You might think I am just talking about Microsoft's quirky implementation but even in the Linux-sphere it isn't consistent see: http://www.greenend.org.uk/rjk/tech/regexp.html http://www.greenend.org.uk/rjk/tech/regexp.html You take a complex format string which was design to use the fewest characters instead of with clarity in mind, you then have every major application and library diverge on basic support and spec for features, and then you have all of them hack on support for UNICODE in their own unique way. Regular Expressions likely won't ever die, but I for one would happily switch to an alternative with better readability, UNICODE support from day zero, and fewer niche features to keep things uniform. I'm tired of re-learning RegEx only to have everything I've learned either be forgot or not work the second I app switch.
- kbenson 9y ago
  For the most part, it's just a matter of knowing if you're using POSIX Basic Regular Expressions (BRE), POSIX Extended Regular Expressions (ERE), or Perl regular expressions. Learn those, or at least the main differences between them, and the vast majority of the regular expression engines in software you use will become more recognizable.

keithnz 9y ago

I love regex. Not just for doing pattern matching in code, but for searching and data transformation in editors and tools. But yes, lots of people do seem to do the "I suck at regex". Even when I notice people do crazy long winded transformations by hand which I then do within seconds. Still doesn't seem enough motivation for them to learn them properly.

zaarn 9y ago

Regex's are awesome as a swiss army knife. Though in my experience, outside some edge cases where the format never changes (like matching a domain name in a URL) a regex is hell to maintain when you come back 3 years later. There is also always the fun of people trying (and failing) to use regex in emails.

ulucs 9y ago
.+@.+ seems to be the only one without too many false-negatives
- zaarn 9y ago
  Well the best solution is to check if .+@(.+) matches and then try to lookup what the capture group returned via your DNS resolver. If it has an MX record (or CNAME to something with MX), then deliver the mail to there. If you can't resole the domain part, return an error.

reificator 9y ago

Regex is great for very simple text processing, and I tend to do a lot of that. The most important parts to get comfortable with are: * Capture groups and alternation * Character sets * Anchors (start and end of line) * Common escape sequences (digit, word, whitespace) * Repeat (any, one or more, n-m) * Common flags (global, multi-line, case-insensitive) If you need more than that, it's time to start evaluating other tools IMO. A lookahead here and there is okay, but I'd avoid them if possible. The best way I found to learn regex was to take a set of inputs that I wanted to match, (and a set that I didn't) and play around on https://regex101.com/ https://regex101.com/ until I got a pattern that did what I wanted. You'll very quickly start to learn the above bulletpoints, and before long you'll be able to write patterns without any reference. If you find that your regexes are getting too large or unwieldy or difficult to understand (despite knowing the above bulletpoints) then you probably need a parser or some other more suitable tool.

DaveChild 9y ago

When I was learning my way around regexes, I found a cheat sheet helped. I made my own - you can download it free from here: https://www.cheatography.com/davechild/cheat-sheets/regular-expressions/ https://www.cheatography.com/davechild/cheat-sheets/regular-...

pmarreck 9y ago

I absolutely unabashedly love Regex. It's like solving a puzzle that ends up eliminating work (by doing said work) extremely concisely. What better kind of puzzle is there?

zombieprocesses 9y ago

> I think they're great though, if you take the time to understand them. Regular expression is useful for sure. What is terrible is that every language, shell and platform has different "styles and implementation". On windows, cmd, powershell, C#, sql server, etc all have their own styles. It's similar enough and at the same time different enough to drive you insane. Throw in linux with their shells, vi(m), perl, etc all using their own variants. But the biggest problem is that regex is prone to "set it and forget it" issue. It's something we use once in a while and forget. Was it brackets or parentheses or braces for defining character ranges? Does . or + signify one or more? And ever try deciphering someone's undocumented multiline regex? Fun times.