9 ms·
Regular Expressions – Mastering Lookahead and Lookbehind
- arthur5005 9y agoYou’ve got a problem you think regex can solve, now you’ve got 2 problems.
- wruza 9y agoI wrote a list of json keys that should be taken from a message and :‘<,’>s/\(\S\+\)\s{0,}/t.\1 = message.\1;\r/g Hey, did you commit already? Still typing?
- jessaustin 9y agoIt looks like your quote characters might be messed up there? Anyway, to parse json on the command line one should just use jq.
- wruza 9y agoSingle quotes were modified by HN engine, right. It is not a command line (thinking of sed?), it is the middle of a source code in my editor. I have a json-parsed message and t is a target object. a b foo c t.a = !!message.a; t.b = String(message.b); t.foo = message.foo; t.c = tonum(message.c);
- filipmandaric 9y agoMost of the time I mention the topic of regular expressions to other developers, I usually hear self-critical commentary like "oh, I'm terrible at regex", and rarely anyone who loves them. I think they're great though, if you take the time to understand them. They're something like a Swiss Army knife for programming.
- BenjiWiebe 9y agoI love them and I think I'm great with them. (Though I'm probably not as great at them as a bunch of you reading this comment)
- jstarfish 9y agoThey're definitely useful, and I can cobble them together to get lots of otherwise tedious and complex parsing tasks done, but when I come back to them a week later I have no idea what the hell the pile of wingding vomit I wrote was supposed to do. I find myself writing simpler ones and tying them together with app code just for sanity's sake.
- testestx 9y ago> I find myself writing simpler ones and tying them together with app code just for sanity's sake. Or just use PEG, parser combinators, or other more readable parsing abstractions
- TeMPOraL 9y agoSome regex implementations allow for comments in the string; if your does not, you can probably make it work with concatenation, like: String pattern = "^https+" // match the protocol at the beginning + "([a-zA-Z])+" // match the machine name + ... Honestly, I use regular expressions because, even in such format expanded with comments, I haven't seen anything more readable after you get used to regex operators. I guess the closest would be the alternative format in CL-PPCRE. For instance: CL-USER> (cl-ppcre:parse-string "\\b\\d{1,3}\\.\\d{1,3}\\.\\d{1,3}\\.\\d{1,3}\\b") (:SEQUENCE :WORD-BOUNDARY (:GREEDY-REPETITION 1 3 :DIGIT-CLASS) #\. (:GREEDY-REPETITION 1 3 :DIGIT-CLASS) #\. (:GREEDY-REPETITION 1 3 :DIGIT-CLASS) #\. (:GREEDY-REPETITION 1 3 :DIGIT-CLASS) :WORD-BOUNDARY) But then, any such form can get mouthful: CL-USER> (cl-ppcre:parse-string "((\\b[0-9]+)?\\.)?\\b[0-9]+([eE][-+]?[0-9]+)?\\b") (:SEQUENCE (:GREEDY-REPETITION 0 1 (:REGISTER (:SEQUENCE (:GREEDY-REPETITION 0 1 (:REGISTER (:SEQUENCE :WORD-BOUNDARY (:GREEDY-REPETITION 1 NIL (:CHAR-CLASS (:RANGE #\0 #\9)))))) #\.))) :WORD-BOUNDARY (:GREEDY-REPETITION 1 NIL (:CHAR-CLASS (:RANGE #\0 #\9))) (:GREEDY-REPETITION 0 1 (:REGISTER (:SEQUENCE (:CHAR-CLASS #\e #\E) (:GREEDY-REPETITION 0 1 (:CHAR-CLASS #\- #\+)) (:GREEDY-REPETITION 1 NIL (:CHAR-CLASS (:RANGE #\0 #\9)))))) :WORD-BOUNDARY)
- paulryanrogers 9y agoWhile useful to some I think advanced RE are like mixing in Perl or playing code golf with production code. They tend to make code harder to read. My preference in such cases is for multiple separated or longer REs (which can be at least split in the surrounding code) and each part named or heavily commented. Of course it's always worthwhile to consider non-RE solutions if the problem can be broken down enough. EDIT: Fixed typo
- filipmandaric 9y agoFair enough, but I really think the benefits of advanced regular expressions are underappreciated in non-production and even non-application contexts. Laypeople (and occasionally even developers) are impressed when you show them how to search through a document or file system using a really complicated pattern, where it would have taken several iterations of data manipulation to achieve the same result without using advanced regular expressions.
- Tobba_ 9y agoIt'd help a lot if the grammar was actually readable. Combinations like .* don't visually "read" like a single unit, and then to make everything worse you often need a crazy amount of backslashes. I'm not sure how you could fix that without introducing completely new characters or color-coding parts of the expression though.
- freedomben 9y agoThe back slashes for escaping are absolutely awful. This is one of the worst things about Java. It's much better in languages with regex literals like Ruby and JavaScript.
- kbp 9y agoIt's especially nicer in Ruby (which got it from Perl) where you can use whatever delimiters you like for regexes, with /abc/, %r"abc", %r{abc}, %r#abc# and so on all being equivalent, so you can just about always pick something that won't clash with the characters in your pattern (You can even use spaces as the delimiters, which looks terrible).
- rdiddly 9y agoThat is truly bizarre... I went looking for info on lookaheads just today, and found myself on that very site, and now it's on HN. It's just that ol' HN magic I guess.
- projektir 9y agoPlugging this website as I've found it very useful to learn simple regex with / get over my "oh God I don't know Regex": https://regexone.com/ https://regexone.com/
- asicsp 9y agoI came to know about this wonderful site when I saw this article - https://www.rexegg.com/regex-best-trick.html https://www.rexegg.com/regex-best-trick.html example: $ # all words except those starting with 'c' or 'C' $ echo 'Car Bat cod12 Map foo_bar' | grep -ioP '\bc\w+(*SKIP)(*F)|\w+' Bat Map foo_bar for more details: https://www.rexegg.com/backtracking-control-verbs.html#skipfail https://www.rexegg.com/backtracking-control-verbs.html#skipf...
- lifthrasiir 9y agoWhich is a fancy way to say `grep -iv '^c'`. EDIT: Oh, I missed that the input was a single line. I personally feel that control verbs are bad additions to the regexp, even though I do know that it is not a big addition to the regexp engine itself (e.g. naturally extended from posesssive quantifiers like `a++` or atomic groups `(?>foo)`). Most uses of such verbs can be expressed with combined parsers and simpler regexps, in the much simpler and maintainable way.
- asicsp 9y agosorry, it is not same as `grep -iv '^c'` the `-o` option allows to output only matching portion, the regex is meant to extract all words other than those starting with 'c' or 'C' here's hopefully better example $ # do something with words not surround by quotes $ echo 'I like "mango" and "guava"' | perl -pe 's/"[^"]+"(*SKIP)(*F)|\w+/\U$&/g' I LIKE "mango" AND "guava"
- lifthrasiir 9y agoOh, you are right. I missed that all words are in the same line. That said even the original article mentions that it only moves the captured group to the entire match; I am generally in a position to avoid all uses of control verbs, especially if it only costs one or probably two lines of the additional code that I can fully control and comprehend.
- sshine 9y agoA use-case for lookarounds that I often use is: grep -Po '(?<=...)pattern' Which also cuts out and prints the relevant part of the line. This saves a trip through cut, awk or perl. (-P is PCRE and -o is print only matched characters, which the lookarounds aren't a part of.)
- asicsp 9y agoI often use `\K` instead, which also helps if it is variable length lookbehind $ echo 'foo=5, Bar=3; x1=83, y=120' | grep -oP '\b[a-z]+=\K\d+' 5 120 further reading: https://stackoverflow.com/questions/11640447/variable-length-lookbehind-assertion-alternatives-for-regular-expressions/11640500#11640500 https://stackoverflow.com/questions/11640447/variable-length...
- JepZ 9y agoI like to add the lookahead: grep -oP '(?<=...)pattern(?=...)' The caveat however is that the look{ahead,behind} pattern has to be of fixed length.
- mnarayan01 9y agoWow. I do this all the time, and the additional step of piping to sed never ceased to annoy me. You've made my day.
- d--b 9y ago> our pattern becomes: > \A(?=\w{6,10}\z)(?=[^a-z][a-z])(?=(?:[^A-Z][A-Z]){3})(?=\D\d). And then these guys wonder why people hate regexes? The "now you have 2 problems" quote fit perfectly for that case.
- pygy_ 9y agoThe regexp syntax was devised for write-only programming at the terminal (at a time when a terminal was a physical object, not a window in your GUI). The regular formalism is pretty neat though. There are alternative syntaxes (e.g. multiline regexps in Python) that are better suited for complex matchers.
- jokoon 9y agoCan you put line returns and indents in a regex? If you escape them it should work, I guess.
- qaq 9y agofor those who find regex not very readable: https://github.com/VerbalExpressions https://github.com/VerbalExpressions // Create an example of how to test for correctly formed URLs var tester = VerEx() .startOfLine() .then('http') .maybe('s') .then('://') .maybe('www.') .anythingBut(' ') .endOfLine();
- pygy_ 9y agohttps://github.com/pygy/compose-regexp.js https://github.com/pygy/compose-regexp.js is another option (800 bytes mingzipped): const {sequence, suffix} = composeRegexp; const maybe = suffix("?"); const oneOrMore = suffix("+"); const urlMatcher = sequence( /^/, "http" maybe("s"), "://", maybe("www."), oneOrMore(/[^ ]/), /$/ );
- nolliebs180 9y ago> It is that at the end of a lookahead or a lookbehind, the regex engine hasn't moved on the string. You can chain three more lookaheads after the first, and the regex engine still won't move. Omg, thank you, that is the insight I needed and now I completely get it. Internet +1 for the day.
- passthejoe 9y agoI used this article about a week ago to help me with a web scrape. Good stuff!
- petrikapu 9y agoIs there regular expression to match regular expression?
- ridiculous_fish 9y agoNo. One way to convince yourself of this is that regexp capture groups must properly nest: /())(()/ is invalid for example. Regexps famously cannot match balanced parenthesis.