4 ms·

Beautiful yet terrifying. I'm glad that regex isn't something I'll ever have to touch.

jagged-chisel 2y ago
The good news is it’s read only- if you’re reading for understanding, you’re doing it wrong :-D
dleeftink 2y ago
It's not that bad. AutoRegex[0] and regex gen [1] make it more accessible than ever. [0]: https://www.autoregex.xyz/ https://www.autoregex.xyz/ [1]: https://regex-generator.olafneumann.org https://regex-generator.olafneumann.org
- janfoeh 2y ago
  Heh, I just fed autoregex a regex from one of my projects, and it simply times out. It comforts me to know that billion dollar LLMs have to chew on those just as much as I do.
  - dleeftink 2y ago
    A proper litmus test, what's causing the hang in your case? Regex to me, is pattern finding and abstraction taken to the extreme. I like the challenge
    - janfoeh 2y ago
      Good question. The regex I tried is for extracting amounts in EUR and USD: / (?<=^|[ \t]) (?<currency_prefix_with_space> (?<currency_prefix> €|EUR|\$|USD ) [ \t]? )? (?<number> (?<integral> -? \d{1,3} (?:[\.,]\d{3}|\d*) ) [\.,] (?<fraction> \d{2,3} ) ) (?<currency_postfix_with_space> [ \t]? (?<postfix_or_ending> (?<currency_postfix> €|EUR|\$|USD ) ) | (?<ending> [ \t]|$|\n ) )/x
      - dleeftink 2y ago
        I'd imagine many nested named capturing groups may trip even the best automated system! I do like the solution though. I would've probably approached it differently, trying to first get the 'inverted' match (i.e. ignore anything that isn't a currency-like pattern) and refine from there. A bit like this one I did a while back, to parse garbled strings that may occur after OCR [0]. I imagine the approach does not translate fully, because it's pattern extraction rather than validation. [0]: https://observablehq.com/@dleeftink/never-go-nuts https://observablehq.com/@dleeftink/never-go-nuts
        janfoeh 2y ago
        Thanks for sharing! I have to admit I do not have the necessary brain cycles to spare today, but OCR processing is indeed of interest to me, and I will take a more in-depth look in the upcoming days. The idea of an exclusionary approach sounds interesting as well. I'll have to think about that a bit.
        dleeftink 2y ago
        Check out WordNinja in case regex doesn't cut it! [0] [0]: https://github.com/keredson/wordninja https://github.com/keredson/wordninja
        janfoeh 2y ago
        Will do, thanks again!
    - tclancy 2y ago
      I think what's causing the hang is using the site. I gave it [a-f0-9]{8}(?:-[a-f0-9]{4}){3}-[a-f0-9]{12} and it sat thinking about things for a few minutes. So I tried the reverse and asked it "Match a UUID" and it sat and thought about things. Now I am enlightened.
      - janfoeh 2y ago
        Aww shucks, and here I was feeling a wee bit proud of myself for managing to break it.
sgc 2y ago
Regex is one of my favorite things to work with. It's surgery. I love having a very specific problem and a very sharp knife. What I hate is chasing down language or library bugs, or undocumented limitations. Of course I would hate it if I had never used it extensively. Its terseness is harder if you only have to pick it up occasionally.
- jpc0 2y ago
  Remember to get a license for your code https://regexlicensing.org/ https://regexlicensing.org/
  - jasonjayr 2y ago
    On the list of disasters, they forgot "parsing html": https://stackoverflow.com/questions/1732348/regex-match-open-tags-except-xhtml-self-contained-tags https://stackoverflow.com/questions/1732348/regex-match-open...
    - deleted 2y ago
      [deleted]
    - joseda-hg 2y ago
      Huh, I tried sharing the Zalgo answer here, but it didn't quite work, never knew HN wouldn't render those kinds of characters
      - qingcharles 2y ago
        It did, but I kinda went over the top with it once and dang had to tell me off, so it might be they added filtering.
        genewitch 2y ago
        o, there's your problem. Only half is supposed to go over the top. The rest is supposed to go below.
sumtechguy 2y ago
I think it comes down to the syntax being used to make regex. It is borderline an assembly language level of making them. Better language tools would make it easier for people to grok. Basically instead of using chars like ?^()$ and the like lets use words people understand.
- aranchelk 2y ago
  It’s not just the syntax. Regex isn’t directly composable (unless you just mash together strings). They cannot define nested structures, I.e. no recursion. They can’t maintain context. We don’t need better language tools. Better parsers can, and already have, been implemented in libraries. I’d have a look at parser combinators.
  - dur-randir 2y ago
    (?R) is very much recursion in there.
    - johnisgood 2y ago
      > They can’t maintain context. Is true though, or perhaps "state"? I know I had to come up with an algorithm because regexp alone couldn't do what I wanted (not even advanced features like lookahead, lookbehind, etc.)
  - davidwritesbugs 2y ago
    I played with parser combinators in Elm and found it very useful c.f. regexes - the issues were mainly down to the Elm implementation. Not sure how effective they are in Rust or Go
- rstuart4133 2y ago
  As @aranchelk said LL and LR grammars are literally that. They allow you to assign names to regexs and compose them. Libraries like python's lrparsing [0] let you assign regex's (aka tokens) to variables, then build grammars by combining them using python expressions. For example: while_statement = Keyword('while') + '(' + expression + ')' + block statement = while_statement | .... block = Token('{') + Repeat(statement + ';') + Token('}') These grammars are more complex, with more rules you have to follow, but also more checking is done when they are compiled. They tend to mostly work once they do compile. So they are what you asked for, but there is no free lunch. On the down side, lrparsing is pure python so it's slower than python's inbuilt regex's. [0] https://lrparsing.sourceforge.net/ https://lrparsing.sourceforge.net/
badc0ffee 2y ago
I'm comfortable with it (having previously been paid to write Perl for several years), but what I don't like is how many other people I have worked with aren't, and just blindly paste in (totally incorrect) regexes from Stack Overflow.
UltraSane 2y ago
Writing one that does exactly what you want is very satisfying. And with Python you can have named groups extracted into a dict and it just feels like magic.

The good news is it’s read only- if you’re reading for understanding, you’re doing it wrong :-D

It's not that bad. AutoRegex[0] and regex gen [1] make it more accessible than ever. [0]: https://www.autoregex.xyz/ https://www.autoregex.xyz/ [1]: https://regex-generator.olafneumann.org https://regex-generator.olafneumann.org

janfoeh 2y ago
Heh, I just fed autoregex a regex from one of my projects, and it simply times out. It comforts me to know that billion dollar LLMs have to chew on those just as much as I do.
- dleeftink 2y ago
  A proper litmus test, what's causing the hang in your case? Regex to me, is pattern finding and abstraction taken to the extreme. I like the challenge
  - janfoeh 2y ago
    Good question. The regex I tried is for extracting amounts in EUR and USD: / (?<=^|[ \t]) (?<currency_prefix_with_space> (?<currency_prefix> €|EUR|\$|USD ) [ \t]? )? (?<number> (?<integral> -? \d{1,3} (?:[\.,]\d{3}|\d*) ) [\.,] (?<fraction> \d{2,3} ) ) (?<currency_postfix_with_space> [ \t]? (?<postfix_or_ending> (?<currency_postfix> €|EUR|\$|USD ) ) | (?<ending> [ \t]|$|\n ) )/x
    - dleeftink 2y ago
      I'd imagine many nested named capturing groups may trip even the best automated system! I do like the solution though. I would've probably approached it differently, trying to first get the 'inverted' match (i.e. ignore anything that isn't a currency-like pattern) and refine from there. A bit like this one I did a while back, to parse garbled strings that may occur after OCR [0]. I imagine the approach does not translate fully, because it's pattern extraction rather than validation. [0]: https://observablehq.com/@dleeftink/never-go-nuts https://observablehq.com/@dleeftink/never-go-nuts
      - janfoeh 2y ago
        Thanks for sharing! I have to admit I do not have the necessary brain cycles to spare today, but OCR processing is indeed of interest to me, and I will take a more in-depth look in the upcoming days. The idea of an exclusionary approach sounds interesting as well. I'll have to think about that a bit.
        dleeftink 2y ago
        Check out WordNinja in case regex doesn't cut it! [0] [0]: https://github.com/keredson/wordninja https://github.com/keredson/wordninja
        janfoeh 2y ago
        Will do, thanks again!
  - tclancy 2y ago
    I think what's causing the hang is using the site. I gave it [a-f0-9]{8}(?:-[a-f0-9]{4}){3}-[a-f0-9]{12} and it sat thinking about things for a few minutes. So I tried the reverse and asked it "Match a UUID" and it sat and thought about things. Now I am enlightened.
    - janfoeh 2y ago
      Aww shucks, and here I was feeling a wee bit proud of myself for managing to break it.

sgc 2y ago

Regex is one of my favorite things to work with. It's surgery. I love having a very specific problem and a very sharp knife. What I hate is chasing down language or library bugs, or undocumented limitations. Of course I would hate it if I had never used it extensively. Its terseness is harder if you only have to pick it up occasionally.

jpc0 2y ago
Remember to get a license for your code https://regexlicensing.org/ https://regexlicensing.org/
- jasonjayr 2y ago
  On the list of disasters, they forgot "parsing html": https://stackoverflow.com/questions/1732348/regex-match-open-tags-except-xhtml-self-contained-tags https://stackoverflow.com/questions/1732348/regex-match-open...
  - deleted 2y ago
    [deleted]
  - joseda-hg 2y ago
    Huh, I tried sharing the Zalgo answer here, but it didn't quite work, never knew HN wouldn't render those kinds of characters
    - qingcharles 2y ago
      It did, but I kinda went over the top with it once and dang had to tell me off, so it might be they added filtering.
      - genewitch 2y ago
        o, there's your problem. Only half is supposed to go over the top. The rest is supposed to go below.

sumtechguy 2y ago

I think it comes down to the syntax being used to make regex. It is borderline an assembly language level of making them. Better language tools would make it easier for people to grok. Basically instead of using chars like ?^()$ and the like lets use words people understand.

aranchelk 2y ago
It’s not just the syntax. Regex isn’t directly composable (unless you just mash together strings). They cannot define nested structures, I.e. no recursion. They can’t maintain context. We don’t need better language tools. Better parsers can, and already have, been implemented in libraries. I’d have a look at parser combinators.
- dur-randir 2y ago
  (?R) is very much recursion in there.
  - johnisgood 2y ago
    > They can’t maintain context. Is true though, or perhaps "state"? I know I had to come up with an algorithm because regexp alone couldn't do what I wanted (not even advanced features like lookahead, lookbehind, etc.)
- davidwritesbugs 2y ago
  I played with parser combinators in Elm and found it very useful c.f. regexes - the issues were mainly down to the Elm implementation. Not sure how effective they are in Rust or Go
rstuart4133 2y ago
As @aranchelk said LL and LR grammars are literally that. They allow you to assign names to regexs and compose them. Libraries like python's lrparsing [0] let you assign regex's (aka tokens) to variables, then build grammars by combining them using python expressions. For example: while_statement = Keyword('while') + '(' + expression + ')' + block statement = while_statement | .... block = Token('{') + Repeat(statement + ';') + Token('}') These grammars are more complex, with more rules you have to follow, but also more checking is done when they are compiled. They tend to mostly work once they do compile. So they are what you asked for, but there is no free lunch. On the down side, lrparsing is pure python so it's slower than python's inbuilt regex's. [0] https://lrparsing.sourceforge.net/ https://lrparsing.sourceforge.net/

badc0ffee 2y ago

I'm comfortable with it (having previously been paid to write Perl for several years), but what I don't like is how many other people I have worked with aren't, and just blindly paste in (totally incorrect) regexes from Stack Overflow.

UltraSane 2y ago

Writing one that does exactly what you want is very satisfying. And with Python you can have named groups extracted into a dict and it just feels like magic.